Two things, formulated slightly different as by the OP:
$$
x(t) = \int_{-\infty}^{\infty} G(t - t')\; F(t')\; dt'.
$$ $$
G(t-t') = \begin{cases} 0 & t' \gt t \\ (t - t')/m & t' \lt t \end{cases}.
$$
Let's work the other way around to check this:
$$
x(t) = \int_{-\infty}^t \frac{(t-t')}{m}\; F(t')\; dt'
$$
Use the Leibniz integral rule
for differentiating under the integral sign:
$$
{\displaystyle {\frac {d}{dx}}\left(\int _{a}^{x}f(x,t)dt\right)=
f{\big (}x,x{\big )}+\int _{a}^{x}{\frac {\partial }{\partial x}}f(x,t)dt,}
\\ \mbox{with} \quad f(t,t')=\frac{(t-t')}{m}\; F(t') \quad \Longrightarrow \\
\dot{x} = \frac{dx}{dt} = 0 + \int_{-\infty}^t \frac{\partial}{\partial t}\frac{(t-t')}{m}\; F(t')\; dt' =
\frac{1}{m}\int_{-\infty}^t F(t')\; dt'
$$ $$
m\,\ddot{x} = F(t)
$$
So it seems that the kernel is correct.
But not very useful, it seems. Take the simplest possible example, with $m=a=\mbox{constant}$:
$$
F = \begin{cases} 0 & t \lt 0 \\ ma & t \gt 0 \end{cases}.\\
x(t) = \int_{-\infty}^t \frac{(t-t')}{m}\; F(t')\; dt'=
a\left[t \int_0^t dt'- \int_0^t t'\; dt'\right]\\x(t) = a\,t^2 - \frac12 a\,t^2 = \frac12 a\,t^2
$$
So the kernel only solves for displacements without initial positions or initial velocities.
EDIT. The more obvious approach is to integrate the Newton force equation directly.
Also take into account that physical events normally start at timestamp $t=0$ instead of $t=-\infty$.
$$
\ddot{x} = \frac{F(t)}{m} \\ \dot{x}(t) = \int_0^t \frac{F(t')}{m}\; dt' \\
x(t) = \int_0^t \dot{x}(t')\; dt'
$$
Now look what happens with partial integration:
$$
\int_0^t \dot{x}(t')\; dt' = \left[t'\dot{x}(t')\right]_{t'=0}^{t'=t} - \int_0^t t'\ddot{x}(t')\; dt'
= t\,\int_0^t \frac{F(t')}{m}\; dt' - \int_0^t t'\,\frac{F(t')}{m}\; dt' \\ \Longrightarrow \quad
x(t) = \int_{-\infty}^{+\infty} G(t-t')F(t')\; dt' \quad \mbox{with} \quad G(t) = H(t)\frac{t}{m}
$$
where it is assumed that $\,F(t\lt 0) = 0\,$; $\,H(t)\,$ is the Heaviside step function.
Hence, apart from some technicalities as mentioned above, the two approaches are the same.
No, it's not too simple. While solving the equation $\,m\ddot{x}=mv\,\delta(t)\,$ for the Green's function,
think of hitting a billiard ball that flies away according to $\,x(t)=H(t)\,v\,t\,$ where $\,H(t)\,$ is the Heaviside step function.