Sorry, I wrote a bit in haste and perhaps my recommendations weren't so clear.
Sensor size almost directly dictates the physical size of the camera and the size of the lens (for particular focal length). That's why a bridge camera like Sony HX300 with a tiny 1/2.3" sensor can have a fairly manageable size 50x zoom on it. On a much larger size 35mm full frame camera (say Nikon D800) the same zoom would be a meter in length and weight c. 15kg or so. Not very pocketable.
The downside with small sensor is that they work with less light, produce more noise and give out less working details once the image is blown up to 1:1 size. But having an image to work with, because you had a camera you could carry, is better than having a perfect image from the huge full frame sensor, that you could never have taken, because you couldn't haul the gear.
My compromise falls between these two sizes: 1/2.4" sensor and full frame 35mm sensor. The size is called APS-C. There are many DSLR cameras in this range. The smallest come from Sony (Nex series) and they still have decent image quality up to ISO 3200 sensitivity. Of course, you won't get the super-long zooms that are available on the aforementioned 50x zoom bridge cameras, but the image quality is better.
So, if I were to buy now, I'd buy a camera that I can carry as often as possible (i.e. you determine the size, based on your needs/limitations) and within the budget that is available.
Then I'd go to dxomark.com sensor database:
DxOMark - Camera Sensor Database
And look for a recent camera in my price/size range that has as high "Sports (Low-Light ISO)" score as possible, unless you are thinking of only shooting in bright daylight, in which case this hardly matters. This determines the amount of noise (or lack of) that you'll get when shooting in less than ideal lighting conditions (say 90% of UFO photos/videos?).
A decent camera to start with in this regard would be Sony NEX-3N.
Then it's finding a maximum zoom lens for that so that you can get the objects closer. Unfortunately, the larger the sensor, the bigger the lenses are. In case of NEX, a 18-200M would give you a decent range (11x zoom factor), but not get objects that are very far.
To shoot objects that are very far away, one needs to go to the 50x zoom range cameras. Out of the current generation (5/2013) the choice are pretty much:
Sony DSX-HX300, 24mm-1200 (50x)
Fujifilm SL1000, 24mm-1200mm (50x)
Canon Powershot SX50 HS, 24mm-1200mm (50x)
They all shoot video too and have optical stabilization (which is a must when shooting at the far end of the zoom range). However, they all have a dinky 1/2.3" sensor, which gives a lot of smudge in detail and plenty of noise. But you can get one for $400-$500, whereas going the dslr body+lens route, one usually has to pay the same for the body alone and another for a decent lens.
So it's all a compromise.
One additional thing I'm experimenting with also myself is removing the infrared frequency filtering piece of glass from the front of the sensor inside the camera. This will totally muck up the colours in any subsequent videos/photos and may even mess a bit with the autofocussing, but it will extend the ability of the camera to shoot into the near-infrared frequencies (say up to 1000 nm or so). If one wants to go further, one can buy a visible light frequency cut-off filter that then lets only near-IR to the camera sensor.
This way it is possible to "see" through the camera things that the normal naked eye cannot see or sees very faintly (in clouds/smog/sky). I've done this modification for one of my videocameras and while all of the ghost hunters also seem to do the same, it hasn't magically transformed me into a super UFO hunter, but it's fun to learn with. However, the downside is that this camcorder is now pretty much a UFO-videocam/art projects type faux colours camera only. Not usable for everyday family stuff anymore, unless your family likes to take a lot of acid...
That's pretty much all I know on this. Not much, but passing it along.
I think the field of ufology would benefit a lot if there were are bit more decent cameras out there in capable hands, although I think that in the end the whole issue will be solved by huge/cheap camera/sensor arrays shooting nightskies 24/7 and then computer analyzing them for changes/movement. I think there's plenty of movement/stuff in the sky, it's just that nobody has time to keep looking at the whole coverage of the sky 24/7. That's why we miss a lot of stuff.