We present a complete architecture for scalable quantum computation with ultracold atoms in optical lattices using optical tweezers focused to the size of a lattice spacing. We discuss three different two-qubit gates based on local collisional interactions. The gates between arbitrary qubits require the transport of atoms to neighboring sites. We numerically optimize the nonadiabatic transport of the atoms through the lattice and the intensity ramps of the optical tweezer in order to maximize the gate fidelities. We find overall gate times of a few 100 μs, while keeping the error probability due to vibrational excitations and spontaneous scattering below 10−3. The requirements on the positioning error and intensity noise of the optical tweezer and the magnetic field stability are analyzed and we show that atoms in optical lattices could meet the requirements for fault-tolerant scalable quantum computing.