I forget how to do integration by parts on a manifold every day of the week. Here's a wiki page to remind me how it should work. See also this MO question for some discussion of boundary.

There's obviously integration by parts on an interval:

$$\int_a^b \phi'(t) \psi(t) dt = [\phi(t)\psi(t)]^b_a - \int_a^b \phi(t) \psi'(t) dt.$$

This is obtained by integrating the product rule $(\phi(t)\psi(t))' = \phi'(t)\psi(t) + \phi(t)\psi'(t)$ and using the fundamental theorem.

More general is Stokes's theorem for an $(n-1)$-form $\omega$ on an $n$-dimenisonal manifold $A$, possibly with boundary.

$$\int_A d \omega = \int_{\partial A} \omega.$$

This recovers the integration by parts above for $A = [a,b]$ and $\omega = \phi(t)\psi(t)$, but is of course more powerful generally.

Let $\Omega$ be the volume form on $A$, let $X$ be any vector field on $A$, let $\nu$ be its metric dual and define $\omega := e^*(\nu) \Omega$. By Stokes's theorem, $$ \int_A d (e^*(\nu) \Omega) = \int_{\partial A} \omega.$$ On the other hand, we recognize a Lie derivative in the left-hand side by the Cartan formula and the fact that $\Omega$ is a top form: $d e^*(\nu) \Omega = \{d, e^*(\nu)\} \Omega = L_X \Omega = (\mathrm{div} X) \Omega$.

Thus we have a first form of the divergence theorem: $$ \int_A (\mathrm{div} X) \Omega = \int_{\partial A} e^*(\nu) \Omega.$$ Write in a frame $e_1, \ldots, e_n$ where $e_n$ is normal to $\partial A$ and $\nu = \nu^i e_i$; $e^*(e_n) \Omega$ is the volume form on $\partial A$ and $e^*(e_j) \Omega = 0$ for $j < n$. Thus $$ \int_A (\mathrm{div} X) \Omega = \int_{\partial A} \nu^n \mathrm{vol}_{\partial A}.$$ But $\nu^n = \langle X, e_n\rangle$ so $$ \int_A (\mathrm{div} X) \Omega = \int_{\partial A} \langle X, e_n \rangle \mathrm{vol}_{\partial A}$$ is the familiar form of the divergence theorem.

Our goal is to derive some result like the following: for any $\alpha, \beta$ sections of a metric bundle with compatible connection and for any vector field $X$ on $M$, $$(\nabla_X \alpha, \beta) = (\alpha, -\nabla_X \beta) + \mathrm{(boundary\ terms)}.$$ Round brackets always mean the $L^2(M)$ inner product, while angle brackets mean the fibrewise product.

The starting point here is the metric compatibility condition $X \langle \alpha, \beta \rangle = \langle \nabla_X \alpha, \beta \rangle + \langle \alpha, \nabla_X \beta \rangle$. Integrating this over $M$ gives immediately that $$(\nabla_X, \alpha, \beta) = (\alpha, -\nabla_X \beta) + \int_M X \langle \alpha, \beta \rangle \mathrm{vol}_M.$$ Thus it suffices to recognize $X \langle \alpha, \beta \rangle \mathrm{vol}_M$ as a total derivative and apply Stokes's theorem. Write $f = \langle a, b\rangle$ and consider $d(f \iota_X \mathrm{vol}) = Xf \mathrm{vol}$ (seen to be true locally and then extended globally). Thus $$\int_M X\langle a,b\rangle \mathrm{vol}_M = \int_{\partial M} f \iota_X \mathrm{vol}_M.$$ Now like in the divergence theorem, the only part of $\iota_X$ that 'survives is the normal direction $\langle X, \nu\rangle \nu$, whence $$\int_M X\langle a,b\rangle \mathrm{vol}_M = \int_{\partial M} \langle a,b\rangle \langle X, \nu\rangle \mathrm{vol}_{\partial M}$$ giving the integration by parts formula $$(\nabla_X, \alpha, \beta) = (\alpha, -\nabla_X \beta) + \int_{\partial M} \langle \alpha, \beta \rangle \langle X, \nu\rangle \mathrm{vol}_{\partial M}.$$ In particular, if any of $X, \alpha, \beta$ are compactly supported then we have the expected integration by parts formula $(\nabla_X, \alpha) = (\alpha, -\nabla_X \beta)$.

On a compact manifold, or on any manifold and operating only on the compactly supported sections, the above integration by parts formula gives that $(\nabla_X, \alpha) = (\alpha, -\nabla_X \beta)$ for all $k$-forms $\alpha, \beta$. In a local frame $e_1,\ldots,e_n$ with coframe $\omega^1,\ldots,\omega^n$ we have the formula for the exterior derivative: $d = e(\omega^i) \nabla_{e_i}$. Thus we may compute its formal adjoint: set $\delta = -e^*(\omega^i) \nabla_{e_i}$ and compute \begin{align*} (d \alpha, \beta) &= \left(e(\omega^i) \nabla_{e_i} \alpha, \beta \right) \\ &= \left(\nabla_{e_i} \alpha, e^*(\omega^i) \beta \right) \\ &= \int_M e_i \langle \alpha, e^*(\omega^i) \beta \rangle - \langle \alpha, \nabla_{e_i} e^*(\omega^i) \rangle \mathrm{vol}_M \\ &= \int_M L_{e_i} (\langle \alpha, e^*(\omega^i) \beta \rangle) - \langle \alpha, e^*(\omega^i) \nabla_{e_i} \rangle + \langle \alpha, e^*(\nabla_{e_i} \omega^i) \mathrm{vol}_M \\ &= \int_M L_{e_i} (\langle \alpha, e^*(\omega^i) \beta \rangle \mathrm{vol}_M) - \int_M \langle \alpha, e^*(\omega^i) \beta \rangle L_{e_i} \mathrm{vol}_M - \int_M \left(\langle \alpha, e^*(\omega^i) \nabla_{e_i} \beta \rangle + \gamma^k_{ii} \langle \alpha, e^*(\omega^k) \beta\rangle\right) \mathrm{vol}_M \\ &= \int_{\partial M} e^*(\omega^i) (\langle \alpha, e^*(\omega^i) \beta \rangle \mathrm{vol}_M) + (\alpha, \delta \beta) - \int_M \langle\alpha, e^*(\omega^i) \beta\rangle (\mathrm{div}(e_i) + \gamma_{kk}^i) \mathrm{vol}_M. \end{align*} On the other hand, the divergence of $e_i$ is $\sum_k (e_i)_{k; k}$, and $(e_i)_{k; k} = \langle \nabla_{e_k} e_i, e_k\rangle = -\gamma^i_{kk}$ so the last term vanishes. As before, the only boundary term with contribution is the unit outward normal one, and we conclude that with $\nu$ the unit outward normal, $$(d\alpha, \beta) = (\alpha, \delta \beta) + \int_{\partial M} \langle \alpha, \iota_{\nu} \beta\rangle \mathrm{vol}_{\partial M}.$$ Probably. Would have to check whether boundary terms arise in the first step where we move $e(\omega^i)$ to the other side too. Certainly in the compactly supported case $(d\alpha,\beta) = (\alpha, \delta \beta)$ holds just fine. Of course, we also have $\delta = d^* = (-1)^{|\alpha| + 1} \star^{-1} d \star$ for $\star$ the Hodge star operator.