Organ motion during radiotherapy is one of the causes of uncertainty in dose delivery creating the need to enlarge the planned target volume (PTV) to guarantee full tumor irradiation. In this work, we investigate the feasibility of using real-time 2D/3D registration for tumor motion tracking during radiotherapy based on purely intensity based image processing, thus avoiding markers or fiducials. X-rays are acquired during treatment at a rate of 5.4 Hz. We iteratively compare each x-ray with a set of digitally reconstructed radiographs (DRR) generated from the planning volume dataset, finding the optimal match between the xray and one of the DRRs. The DRRs are generated using a ray-casting algorithm, implemented using general purpose computation on graphics hardware (GPGPU) for best performance. Validation is conducted offline using a phantom and five clinical patient data sets. The phantom motion is measured with an RMS error of 2.1 mm and mean registration time is 220 ms. For the patient data sets, a sinusoidal movement that clearly correlates to the breathing cycle is seen. Mean registration time is always under 105 ms which is well suited for our purposes. These results demonstrate that real-time organ motion monitoring using image based markerless registration is feasible.