Linear algebra is the branch of mathematics concerning linear equations such as
linear maps such as the following:
and their representations in vector spaces and through matrices.
Linear algebra is central to almost all areas of mathematics. For instance, linear algebra is fundamental in modern presentations of geometry, including for defining basic objects such as lines, planes and rotations. Also, functional analysis, a branch of mathematical analysis, may be viewed as the application of linear algebra to function spaces.
Linear algebra is also used in most sciences and fields of engineering, because linear structures are natural models for many physical phenomena; it's also useful in many areas of computing, and is thus applied to most sciences. For nonlinear systems, linear algebra is often used for dealing with first-order approximations: the derivative of a manifold at a point gives the slope of a line which models the manifold's local behavior, and can itself be analyzed with linear algebra.
History
The procedure (using counting rods) for solving simultaneous linear equations now called Gaussian elimination appears in the ancient Chinese mathematical text Chapter Eight: Rectangular Arrays of The Nine Chapters on the Mathematical Art. Its use is illustrated in eighteen problems, with two to five equations.
Systems of linear equations arose in Europe with the introduction in 1637 by René Descartes of coordinates in geometry. In fact, in this new geometry, now called Cartesian geometry, lines and planes are represented by linear equations, and computing their intersections amounts to solving systems of linear equations.
The first systematic methods for solving linear systems used determinants and were first considered by Leibniz in 1693. In 1750, Gabriel Cramer used them for giving explicit solutions of linear systems, now called Cramer's rule. Later, Gauss further described the method of elimination, which was initially listed as an advancement in geodesy.
"The leading mathematicians of the nineteenth and early twentieth centuries developed and used most of the basic results and techniques of linear algebra for perhaps a hundred years, without even defining an abstract vector space: nor is there anything to suggest they suffered thereby."Ian Stewart and David Tall (1979) Algebraic Number Theory, page x (preface) In 1844 Hermann Grassmann published his "Theory of Extension" which included foundational new topics of what is today called linear algebra. In 1848, James Joseph Sylvester introduced the term matrix, which is Latin for womb.
Linear algebra grew with ideas noted in the complex plane. For instance, two numbers and in have a difference , and the line segments and are of the same length and direction. The segments are equipollent. The four-dimensional system of quaternions was discovered by W.R. Hamilton in 1843.Koecher, M., Remmert, R. (1991). Hamilton’s Quaternions. In: Numbers. Graduate Texts in Mathematics, vol 123. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-1005-4_10 The term vector was introduced as representing a point in space. The quaternion difference also produces a segment equipollent to . Other hypercomplex number systems also used the idea of a linear space with a basis.
Arthur Cayley introduced matrix multiplication and the inverse matrix in 1856, making possible the general linear group. The mechanism of group representation became available for describing complex and hypercomplex numbers. Crucially, Cayley used a single letter to denote a matrix, thus treating a matrix as an aggregate object. He also realized the connection between matrices and determinants and wrote "There would be many things to say about this theory of matrices which should, it seems to me, precede the theory of determinants".
Benjamin Peirce published his Linear Associative Algebra (1872), and his son Charles Sanders Peirce extended the work later.Benjamin Peirce (1872) Linear Associative Algebra, lithograph, new edition with corrections, notes, and an added 1875 paper by Peirce, plus notes by his son Charles Sanders Peirce, published in the American Journal of Mathematics v. 4, 1881, Johns Hopkins University, pp. 221–226, Google Eprint and as an extract, D. Van Nostrand, 1882, Google Eprint.
The telegraph required an explanatory system, and the 1873 publication by James Clerk Maxwell of A Treatise on Electricity and Magnetism instituted a field theory of forces and required differential geometry for expression. Linear algebra is flat differential geometry and serves in tangent spaces to manifolds. Electromagnetic symmetries of spacetime are expressed by the Lorentz transformations, and much of the history of linear algebra is the history of Lorentz transformations.
The first modern and more precise definition of a vector space was introduced by Peano in 1888; by 1900, a theory of linear transformations of finite-dimensional vector spaces had emerged. Linear algebra took its modern form in the first half of the twentieth century when many ideas and methods of previous centuries were generalized as abstract algebra. The development of computers led to increased research in efficient algorithms for Gaussian elimination and matrix decompositions, and linear algebra became an essential tool for modeling and simulations.
Vector spaces
Until the 19th century, linear algebra was introduced through systems of linear equations and matrices. In modern mathematics, the presentation through vector spaces is generally preferred, since it is more synthetic, more general (not limited to the finite-dimensional case), and conceptually simpler, although more abstract.
A vector space over a field (often the field of the real numbers or of the complex numbers) is a set equipped with two binary operations. Elements of are called vectors, and elements of F are called scalars. The first operation, vector addition, takes any two vectors and and outputs a third vector . The second operation, scalar multiplication, takes any scalar and any vector and outputs a new . The axioms that addition and scalar multiplication must satisfy are the following. (In the list below, and are arbitrary elements of , and and are arbitrary scalars in the field .)
Axiom Signification Associativity of addition Commutativity of addition Identity element of addition There exists an element in , called the zero vector (or simply zero), such that for all in . Inverse elements of addition For every in , there exists an element in , called the additive inverse of , such that Distributivity of scalar multiplication with respect to vector addition Distributivity of scalar multiplication with respect to field addition Compatibility of scalar multiplication with field multiplication Identity element of scalar multiplication , where denotes the multiplicative identity of .
The first four axioms mean that is an abelian group under addition.
The elements of a specific vector space may have various natures; for example, they could be tuples, sequences, functions, polynomials, or matrices. Linear algebra is concerned with the properties of such objects that are common to all vector spaces.
Linear maps
Linear maps are mappings between vector spaces that preserve the vector-space structure. Given two vector spaces and over a field , a linear map (also called, in some contexts, linear transformation or linear mapping) is a map
that is compatible with addition and scalar multiplication, that is
for any vectors in and scalar in .
An equivalent condition is that
for any vectors in and scalars in .
When are the same vector space, a linear map is also known as a linear operator on .
A bijective linear map between two vector spaces (that is, every vector from the second space is associated with exactly one in the first) is an isomorphism. Because an isomorphism preserves linear structure, two isomorphic vector spaces are "essentially the same" from the linear algebra point of view, in the sense that they cannot be distinguished by using vector space properties. An essential question in linear algebra is testing whether a linear map is an isomorphism or not, and, if it is not an isomorphism, finding its range (or image) and the set of elements that are mapped to the zero vector, called the kernel of the map. All these questions can be solved by using Gaussian elimination or some variant of this algorithm.
Subspaces, span, and basis
The study of those subsets of vector spaces that are in themselves vector spaces under the induced operations is fundamental, similarly as for many mathematical structures. These subsets are called linear subspaces. More precisely, a linear subspace of a vector space over a field is a subset of such that and are in , for every , in , and every in . (These conditions suffice for implying that is a vector space.)
For example, given a linear map , the image of , and the inverse image of (called kernel or null space), are linear subspaces of and , respectively.
Another important way of forming a subspace is to consider linear combinations of a set of vectors: the set of all sums
where are in , and are in form a linear subspace called the span of . The span of is also the intersection of all linear subspaces containing . In other words, it is the smallest (for the inclusion relation) linear subspace containing .
A set of vectors is linearly independent if none is in the span of the others. Equivalently, a set of vectors is linearly independent if the only way to express the zero vector as a linear combination of elements of is to take zero for every coefficient .
A set of vectors that spans a vector space is called a spanning set or generating set. If a spanning set is linearly dependent (that is not linearly independent), then some element of is in the span of the other elements of , and the span would remain the same if one were to remove from . One may continue to remove elements of until getting a linearly independent spanning set. Such a linearly independent set that spans a vector space is called a basis of . The importance of bases lies in the fact that they are simultaneously minimal-generating sets and maximal independent sets. More precisely, if is a linearly independent set, and is a spanning set such that , then there is a basis such that .
Any two bases of a vector space have the same cardinality, which is called the dimension of ; this is the dimension theorem for vector spaces. Moreover, two vector spaces over the same field are isomorphic if and only if they have the same dimension. p. 82, §3.59
If any basis of (and therefore every basis) has a finite number of elements, is a finite-dimensional vector space. If is a subspace of , then . In the case where is finite-dimensional, the equality of the dimensions implies .
If and are subspaces of , then
where denotes the span of . p. 23, §1.45
Matrices
Matrices allow explicit manipulation of finite-dimensional vector spaces and linear maps. Their theory is thus an essential part of linear algebra.
Let be a finite-dimensional vector space over a field , and be a basis of (thus is the dimension of ). By definition of a basis, the map
is a bijection from , the set of the sequences of elements of , onto . This is an isomorphism of vector spaces, if is equipped with its standard structure of vector space, where vector addition and scalar multiplication are done component by component.
This isomorphism allows representing a vector by its inverse image under this isomorphism, that is by the coordinate vector or by the column matrix
If is another finite dimensional vector space (possibly the same), with a basis , a linear map from to is well defined by its values on the basis elements, that is . Thus, is well represented by the list of the corresponding column matrices. That is, if
for , then is represented by the matrix
with rows and columns.