Misundersanding

Because I met one of an intimate foreign friend from France on last Saturday, just I said, "Where to?", then he replied, "To work." with his face changing into unpleasant. He must thought that why do I ask such an impolite question invading private his area, I'm not his parent nor a police, it's up to him and no obligation to explain where he is going at that time in our democratic society not a social country in Japan. Don't monitor him! We Japanese often say, "Where to on fine day--- It's good and have a fun, take care and come earlier and safe!", this kind of conversation is very common between neighbors and intimate friends. So, this kind of misunderstandings might happen between foreigners because of the difference of cultural backgrounds and everyday practice. I must apologize him and explain well about our perceptions for sliding fitting.