With over 99.8 percent of the vote, Castillo leads with 50.2 percent against Fujimori's 49.8.
The right-wing candidate Fujimori demands that the votes from over 800 polling stations, corresponding to 200,000 ballot papers, be annulled.
Morales congratulates
The country's election commission Onpe has not yet wanted to announce a winner, but Castillo says that according to his own observers, it is clear that he will win.
He also thanks for congratulations from foreign governments and embassies.
But no country's government has yet officially recognized Castillo as the winner, although Bolivia's former left-wing president Evo Morales has sent congratulations via Twitter.
Castillo led during the night towards Thursday Swedish time with about 67,000 votes, but as the count is still ongoing and votes are questioned by both sides, Onpe predicts that the final result will be delayed.
Maybe as long as ten to twelve days.
Many foreign voices
Just as in previous presidential elections in Peru, the counting of votes drags on.
This is partly due to the fact that ballot papers are to be transported from remote rural areas to the capital Lima as well as from abroad.
One million of the country's eligible voters live abroad.
Fujimori has taken most of the counted foreign votes, but Castillo is very popular with voters in rural areas.
And it will be difficult for Fujimori to catch up, says analyst Hugo Otero.
- Because there are probably more votes left to count in Peru than abroad, he says.
Late Wednesday night local time, Castillo's supporters gathered outside Onpe's polling station in central Lima, while a demonstration for Fujimori was held in a park in the city.
Fujimori supporters claim there has been electoral fraud.