It arises from the Sanskrit root √bṛh (बृह्) meaning to make big or strong, to grow, to increase, to expand, to further (and also to speak). Complex internal sandhi changes the root to 'brah'
Then it could have become brahman in two ways:
a) the suffix 'mant' is added to it, but in that case it would be brahmaan. (mant has the sense of possessing the quality of the stem that precedes it. Like Shaktimaan)
b) the suffix 'man' or 'an' have been added to it, making it a noun. This is a standard method of nominalisation in Sanskrit.
In both cases it is most likely that a commonly used form emerged which superseded its original grammatical form and etymological meaning by a process called Analogy*.
This analysis is speculative based on what little Sanskrit I know. I might be wrong - so It would be great some feedback if someone knows better!
* (Analogy (from Greek ἀναλογία, analogia, "proportion") is a cognitive process of transferring information or meaning from a particular subject (the analogue or source) to another particular subject (the target), and a linguistic expression corresponding to such a process.)