Qubit is the quantum computing equivalent of a bit.
A classical bit can be 0 or 1. There are no probabilities involved at all. If a bit is 1, then it's 1 with 100% probability and 0 with 0% probability. Conversely if a bit is 0, then it's 1 with 0% probability and 0 with 100% probability.
A qubit is somewhat similar. It also has two states: $\lvert 0\rangle$ and $\lvert 1\rangle$. Except now, the total probability is distributed between those two. A qubit is in both states at the same time: a small amount in state $\lvert 0\rangle$ and a small amount in state $\lvert 1\rangle$. A qubit $\lvert\psi\rangle$ is written in the following form.
$$\lvert\psi\rangle = a\lvert 0\rangle + b\lvert 1\rangle$$
But if this wasn't weird enough, the probabilities $a$ and $b$ are not percentages. Rather, they are complex numbers. For example, a qubit might be in a state $\lvert 0\rangle$ with probability amplitude $a = \frac{\sqrt 2}{2} + i\frac{\sqrt 2}{2}$, and in state $\lvert 1\rangle$ with probability amplitude $b = \frac{\sqrt 2}{2} - i\frac{\sqrt 2}{2}$, all at the same time. The probability amplitudes evolve over time according to the Schrodinger equation. For eample, $a$ and $b$ can rotate or exchange magnitudes over time.
A qubit state continues to evolve as this weird object of two complex probabilities amplitudes – until it is measured. When a qubit is measured, it collapses to either $\lvert 0\rangle$ or $\lvert 1\rangle$ with 100% probability. At that point, there are no longer any probabilities or complex numbers inolved. The qubit behaves exactly like a classical bit and we might as well refer to $\lvert 0\rangle$ as 0, and $\lvert 1\rangle$ as 1.
But how do we know whether a qubit is going to collapse to 0 or 1 if $a$ and $b$ are complex numbers rather than percentages? The answer is given by the Born rule. The Born rule states that the real probability of measurement yielding a certain result is proportional to the square of the probability amplitude associated with that result.
$$P_0 = |a|^2 \qquad P_1 = |b|^2$$
We do, however, require the qubit always exists as at least something with 100% probability when we measure it i.e. a qubit can't half exist or exist with 150% probability. Therefore the square magnitudes must add up to 100% which in probability theory is written as $1$. This is also called the second axiom of probability:
$$|a|^2 + |b|^2 = 1$$
A qubit has two basis states, $\lvert 0\rangle$ and $\lvert 1\rangle$. This notation is called the ket-notation. It's a way to give a name to the basis states. The names inside kets have no meaning. We could have just as easily said $\lvert\text{cat}\rangle$ and $\lvert\text{dog}\rangle$. What's important is that there are exactly two distinct quantum states. These are the states we associate complex probability amplitudes to. If we had more than two states, we'd have a qudit. A qubit is just a special case when the number of states is $d = 2$.
We've seen that the qubit state $\lvert\psi\rangle$ is described by two complex numbers $a,b\in\mathbb C$. We might as well write it as a column vector in $\mathbb C^2$, since this will lets us take advantage of linear algebra. The space of all possible states is called a Hilbert space. In this case, Hilbert space is the space of all possible column vectors in $\mathbb C^2$. $$ \lvert\psi\rangle = \begin{pmatrix}a\\b\end{pmatrix},\quad\lvert\psi\rangle\in\mathbb C^2$$
We can then write the column vector $\lvert\psi\rangle$ in the following form. In which case, we recognize the familiar expression $\lvert\psi\rangle = a\lvert 0\rangle + b\lvert 1\rangle$:
$$\lvert\psi\rangle = a\begin{pmatrix}1\\0\end{pmatrix} + b\begin{pmatrix}0\\1\end{pmatrix}$$
So, when it comes to a qubit, everything just boils down to 2-dimensional linear algebra over complex numbers. Basis states are then just special type of vectors in this space:
$$\lvert 0\rangle = \begin{pmatrix}1\\0\end{pmatrix},\quad\lvert 1\rangle = \begin{pmatrix}0\\1 \end{pmatrix}$$
In fact, in Hilbert space $\mathbb C^2$, all kets $\lvert \psi\rangle$ are column vectors, all bras $\langle \phi\lvert$ are just row vectors, and all operators $H$ are just 2×2 matrices (the elements are, of course, complex numbers). This comes from the fact a qubit only has two states and the Hilbert space $\mathbb C$ was enough to capture all possible states.
In quantum mechanics more broadly, there may be infinite number of states. In that case, states are not 2-dimensional vectors, but rather infinite dimensional. At that point, $\mathbb C^2$ is not enough to capture all possible states, so other kinds of Hilbert spaces are used.
A qubit is in a pure state when it's in one of the basis states with 100% probability. When we say 100% probability we mean a complex probability amplitude $1 + i0$. Conversely, when we say 0% probability we mean a complex probability amplitude $0 + i0$.
Because there are only two basis states, and a pure state is when a qubit is in one of the two basis states, there are also only two pure states:
$$ \lvert\psi\rangle = \underbrace{\left(1 + i0\right)}_{a}\cdot \lvert 0\rangle + \underbrace{\left(0 + i0\right)}_{b}\cdot\lvert 1\rangle$$
$$ \lvert\psi\rangle = \underbrace{\left(0 + i0\right)}_{a}\cdot\lvert 0\rangle + \underbrace{\left(1 + i0\right)}_{b}\cdot\lvert 1\rangle$$
A qubit is in a mixed state when total probability distributed among states.
$$ \lvert\psi\rangle = \underbrace{\left(\frac{1}{2} + i0\right)}_{a}\cdot \lvert 0\rangle + \underbrace{\left(\frac{1}{2} + i0\right)}_{b}\cdot \lvert 1\rangle$$
$$ \lvert\psi\rangle = \left(\frac{1}{\sqrt{2}} + i 0 \right)\cdot\lvert 0\rangle + \left(\frac{1}{\sqrt{2}} + i\frac{1}{\sqrt{2}}\right)\cdot \lvert 1\rangle$$
When a qubit is measured the state collapses to a pure state. In both of these examples, the probabilities are equally distributed. Measuring these qubits will yield either state with 50% probability.
A qubit is in an invalid state when the square magnitutdes of its probability amplitudes don't add up to $1$. This is not allowed by quantum mechanics so a qubit will never find itself in an invalid state.
An example is a qubit that's in both states with 100% probability, like $\lvert\psi\rangle = \lvert 0\rangle + \lvert 1 \rangle$. Expanding the notation to see the structure more clearly: $$ \lvert\psi\rangle = \left(1 + 0i\right)\lvert 0\rangle + \left(1 + 0i\right)\lvert 1\rangle$$
When we sum the square magnitudes we get the result $2$ $$ |1 + 0i|^2 + |1 + 0i|^2 = 1^2 + 1^2 = 2$$
The interpretation of this result is that the qubit exists with a total probability of 200%, which doesn't make sense. The total probability distribution among all states must always add up to 100%.
The equation that governs quantum mechanics is the Shrodinger equation. In the most general case it's written in the following form.
$$i\hbar\frac{\partial}{\partial t}\lvert \psi\rangle = H\lvert\psi\rangle$$
The ket $\lvert \psi\rangle$ is a quantum state. Mathematically, it's a vector in a Hilbert space. The operator $H$ is called a Hamiltonian. Mathematically, it's a linear Hermitian opeartor acting on vectors in that space. The Hamiltonian represents the total energy of the system (kinetic energy + potential energy).
In quantum mechanics, there is a postulate that the system should be fully described its state $\lvert\psi\rangle$. This is not a law of nature. Rather it's our responsibility to make that so, by choosing a Hilbert space that appropriately captures all possible states. Often it's possible to combine Hilbert spaces with a tensor product to make a more general Hilbert space that fits our needs.
When the Hilbert space is chosen, one identifies a Hamiltonian that captures the energies of the system. This is done either by looking (e.g. the spherical symmetry in the Hydrogen atom) or just by constructing one that satisfies being a linear Hermitian operator. The Hamiltonian is then plugged in the Schrodinger equation as solved for state $\lvert\psi\rangle$.
Let's take a particle like an electron inside a hydrogen atom. A postulate in quantum mechanics is that an electron should be fully described by its state $\lvert\psi\rangle$. We need a Hilbert space that captures all possible states of an electron. But first, what is an electron state?
An electron is an object in 3D space. Like with qubit, its probability amplitudes are complex numbers. Therefore a state should be something that associates a complex number $\mathbb C$ with every point in space $\mathbb R^3$. So we'll define the electron state as a function $\lvert\psi\rangle: \mathbb R^3\mapsto\mathbb C$.
We also need the electron to exist somewhere in the universe. According to the Born rule, the probability of finding an electron at a point $(x, y, z)$ is proportional to the square magnitude of the complex number associated with that point. Therefore, if we integrate square magnitude of the function across all of 3D space we should get $1$.
So, to capture all possible electron states, we choose the Hilbert space to be a space of all square-integrable functions. This is compactly written as $L^2(\mathbb R^3)$.
$$ \lvert\psi\rangle\in \left\{ f:\mathbb R^3\mapsto\mathbb C \text{ such that } \int_{\mathbb R^3} |f(x)|^2 \mathrm dx = 1\right \} \quad \text{ same as } \quad \lvert\psi\rangle\in L^2(\mathbb R^3)$$
When it comes to the Hamiltonian, we need a Hermitian linear operator acting on vectors in our Hilbert space. It's out of the scope of this section, but it can be proven that if we take the classical equation for energy $E = \frac{p^2}{2m} + V$, and promote $p\mapsto -i\hbar\nabla^2$ and $V\mapsto V$ to operators in Hilbert space we get:
$$H = -\frac{\hbar^2}{2m}\nabla^2 + V$$
The Schrodinger equation becomes: $$-i\hbar\frac{\partial}{\partial t}\lvert\psi\rangle = \left(-\frac{\hbar^2}{2m}\nabla^2 + V\right)\lvert\psi\rangle$$
If we let the potential $V$ be the Coulomb potential $V = e^2/4\pi\varepsilon_0r$ created by the presence of a proton, solving the Schrodinger equation yields the equation for electron orbitals in 3D space.
Let's turn back to the qubit. A postulate in quantum mechanics is that a qubit should be fully described by its state $\lvert\psi\rangle$. We need a Hilbert space that captures all possible states of a qubit. But what is a qubit state?
Unlike the electron, qubit is quite limited in its degrees of freedom: instead of being distributed across all of 3D space, there are only two places the probability amplitudes can go to. We can do something similar to an electron and define a qubit state to be a function $\lvert\psi\rangle: \{0, 1\}\mapsto\mathbb C$. But this is isomorphic to the column vector $\mathbb C^2$. So rather, it's better if we define the state to be a column vector $\lvert\psi\rangle\in\mathbb C^2$. This lets us take advantage of linear algebra.
So, to capture all qubit states, we choose the Hilbert space to be the 2-dimensional complex vector space $\mathbb C^2$. $$ \lvert \psi\rangle\in \left\{\begin{pmatrix}a \\ b\end{pmatrix}\text{ where } a, b\in \mathbb C \text{ and } |a|^2 + |b|^2 = 1 \right \} \text{ same as } \lvert\psi\rangle\in\mathbb C^2$$
Now we need a Hamiltonian. The Hamiltonian is a bit odd. It encodes the energy of the system – but what is the kinetic and potential energy of a qubit? The truth is, qubit doesn't have those. Kinetic and potential energies arise when a system has spatial degrees of freedom. A qubit has two internal degrees of freedom instead. A Hamiltonian can still be defined, just not in terms of kinetic and potential energies.
Mathematically, Hamiltonian is a linear hermitian operator over Hilbert space. Our Hilbert space is $\mathbb C^2$ so linear operators are complex 2×2 matrices. To satisfy being a Hamiltonian, we just need our 2×2 matrix to be Hermitian. A Hermitian matrix is one where if you do a transpose and apply complex conjugation to all its elements it stays the same. A general 2×2 hermitian matrix for some real numbers $x,y,z,w\in\mathbb R$ is:
$$H = \begin{pmatrix}x & z - iw \\ z + iw & y \end{pmatrix}$$
When we plug this in the Schrodinger equation we get the following expression:;
$$i\hbar\frac{\partial}{\partial t}\begin{pmatrix}a(t)\\b(t)\end{pmatrix} = \begin{pmatrix}x & z - iw \\ z + iw & y \end{pmatrix}\cdot\begin{pmatrix}a(t)\\b(t)\end{pmatrix}$$
This is a first order differential equation with respect to time, so when we solve it for the column vector (a, b):
$$\begin{pmatrix}a(t)\\b(t)\end{pmatrix} = e^{iH(t)/\hbar}\cdot\begin{pmatrix}a(0)\\b(0)\end{pmatrix}$$
Notice $H$ is in an exponential function $e^x$, yet $H$ is a 2×2 matrix. How do we define an exponential function of a matrix? We can do it by defining $e^x$ via its Maclaurin series $e^x = 1 + x + x^2 / 2 + \cdots$.
$$e^H = 1 + H + \frac{H^2}{2} + \frac{H^2}{6} + \cdots$$
A classical bit only has two states, 0 and 1. Therefore you can easily picture a classical bit as an on-off switch.
A qubit is a little more complex. However we've seen it consists of two probability amplitudes $\alpha$ and $\beta$, both of which are complex numbers. According to the Schrodinger equation their phases evolve in opposite directions (both at the same angular frequency), but their amplitudes do not change.
Therefore, you can picture a qubit like an analog clock with two hands: One hand spinning clockwise and the other hand spinning counter-clockwise. They're both spinning at the same speed, just in the opposite direction. One is shorter and the other one is longer. The area of the circle they sweep represents the probability. Area of a circle is proportional to the radius squared $\pi r^2$. This matches the Born rule that that the probability of collapsing a qubit in a given state is given by magnitude squared $P(0) = |\alpha|^2$ and $P(1) = |\beta|^2$.