
You should undertake them if you're interested in winning the relevant competition, but should always consider the other opportunities for increasing your score as well. The projects have fixed Production costs and are only unlocked while the relevant competition lasts.
These become available in all cities only after a resolution to start a Scored Competition in the World Congress has been enacted, and turn a city's Production toward increasing the player's score for the competition.
Gathering Storm adds a third type of project: competition projects. District projects will not award great people points if the Great People class is banned by the World Congress Patronage resolution. District projects are a decent way to boost some specific yields for a time (city production is converted each turn into the project yield type at a rate of 15%), and also to accelerate your progress towards attracting a coveted Great Person (when completed most district projects provide 15% of the project cost as great people points). The Production cost of district projects scales up from a base price based on the number of technologies and/or civics a player has discovered, but this also means that their overall effectiveness goes up (as the points they produce are a function of their Production cost). They can be undertaken as many times as a player wants in any city that has the requisite district. They convert a city's Production into other kinds of yields while the project is active and (usually) provide points toward earning Great People when it ends.
District projects, which are closely related to the new districts and Great People. You should undertake these projects only on a need-to-do basis otherwise, you will be wasting Production capacity. They may (usually) be undertaken in all your cities (with the exceptions of the Science Victory projects, which always require an active Spaceport in the city), and usually need be completed only once.
They are required to unlock some powerful feature, such as nuclear weapons, or perform some empire-wide task, such as launching spacecraft.
City projects, which are more or less the same as in previous games. There are now three main types of projects: In Civilization VI and its expansions, a project can be undertaken in the Production queue of a city to provide some sort of benefit which doesn't include creating a permanent feature like a unit, building, or District.