Officials in Saudi Arabia say another 863 people were also injured.
It happened at a holy site called Mina, where two million people go to throw stones at pillars representing the devil.
Similar tragedies have happened at the Hajj before.
What is Hajj?
Image captionMuslim pilgrims walk on roads as they head to cast stones at pillars symbolising Satan
Image captionThere are more than 100,000 air-conditioned tents in Mina for the three million pilgrims
Millions of Muslims travel from around the world to go to Saudi Arabia to complete the Hajj pilgrimage in the holy city of Mecca.
It is the fifth and final pillar of Islam and takes place in the 12th month of the Islamic calendar.
Why do so many people go?
It is a spiritual pilgrimage that every adult Muslim must perform at least once in their lives, if they can afford it and are physically able.
The journey is central to the faith.
Image captionPilgrims spend five days praying both in Mecca and the surrounding desert
What is the stoning of the devil?
Pilgrims travel to Mina, a large valley about 5km (3 miles) from Mecca, where they throw 49 stones over three days at three pillars called jamarat, which represent the devil.
It houses more than 160,000 tents where pilgrims spend the night before the event.
Why have so many people been killed before?
There have been numerous tragedies at Hajj over the years, but this latest stampede is the deadliest in 25 years.
It seem accidents happen because of the sheer amount of people that turn up.
Image captionPeople arrive to throw pebbles at the Jamarat for the stoning of Satan
What happens next?
The Saudi authorities have spent billions of dollars on improving transport and other infrastructure in the area in an attempt to try to prevent more accidents.
Image captionThe jamarat pillars were replaced with walls in 2004 because people were throwing stones at each other by mistake
Isn’t it all older people who go?
No. There are lots of photos on Instagram of young people there under #hajj.
Thousands of British people travel to the pilgrimage each year after saving for the “once in a lifetime trip” spending between £4,000-£5,000.
The tragedy happened during Eid al-Adha, which is known as the Feast of the Sacrifice as it recalls Abraham’s willingness to sacrifice his son in obedience to Allah.