Today the morning subway paper Metro had a similar graphic as above (from here), showing that over 20,000 North Koreans have fled to South Korea, with 19,000 of them coming since 1999. According to the article in Metro*, 607 came before 1990, 488 came during the 1990s, 312 in 2000, and 583 in 2001. I remember around 2004 or 2005 the unification ministry under Roh Moo-hyun said that it wanted to cut back on the number of refugees coming into Korea, which might explain the dip in 2005. It's obviously been steadily increasing since then, but I would have imagined it to be more than 20,000 people, which is statistically a tiny part of the population.
*Update: The graph I saw in Metro is here, while the Segye Ilbo has a graph showing how the percentage of females among North Korean refugees has changed over the years (and is now at 77%):