Each of these officials is aided by officers of the appropriate branches of state government.
Most districts have a distinct headquarters; Mumbai is an example of a city which, despite coming under a district, does not have a district headquarters, though it does have a Collector.
The majority of districts are named after their administrative center. Some are referred to by two names, a traditional one and one that uses the name of the town that is the headquarters. Since most of the districts are named after a town, the word "district" or "District" is appended to distinguish between the town and the district. Official websites very often use District with a capital D in this context.