Understanding urban dynamics is of fundamental importance for the efficient operation and sustainable development of large cities. In this paper, we present a comprehensive study on characterizing urban dynamics using the large scale taxi data in New York City. The pick-up and drop-off locations are firstly analyzed separately to reveal the general trip pattern across the city and the existence of unbalanced trips. The inherent similarities among taxi trips are further investigated using the two-step clustering algorithm. It builds up the relationship among detached areas in terms of land use types, travel distances and departure time. Moreover, human mobility pattern are inferred from the taxi trip displacements and is found to follow two stages: an exponential distribution with short trips and a truncated power law distribution for longer trips. The result indicates that the taxi trip may not fully represent human mobility and is heavily affected by trip expenses and the urban form and geography.