Good question. I am not sure if chromecast would support an and-hoc network (android to TV direct).
You can buy a very cheap wifi router for <£10 second hand on Ebay for example and try it.
You would only be able to cast local videos etc as you would not have access to internet via router.
Some internet streaming may be possible using chromecast friendly apps e.g. youtube over a 3G network (if you have a good dataplan)?