The empire came under attack from the powers it encountered as the borders expanded. From this period onwards they were increasingly weakened by the ongoing wars to defend the borders.
The Angles, Saxons, Franks, from North Europe attacked Northern France and England. From the East, the Huns, Vandals and Visigoths penetrated France.
By 300AD the weakened Roman empire began to have civil wars. The empire was partitioned into two in 285AD and into 3 by 330AD. Ongoing power struggles further weakened the empire. The empire was reunited in 390AD, but was unable to stand the onslaught of the invaders.
British Celts invaded Brittany around 400AD.
The Vandals controlled West France by 420AD, and by negotiation handed control to their allies, the Visigoths. In 450AD an attack by the Huns was repelled.
By 460AD, the Western Roman empire was only a shrunken shadow of its former self with a few isolated remnants.
The Franks expelled the Visigoths and controlled virtually all of present day France by 470AD, with the exception of isolated cities and part of North France. By 500AD France was virtually free of Rome.
The Franks established the Carolingian dynasty and, under Charlemagne, went on to occupy North Italy by 774AD.
They were weakened by continued defensive wars against invading Vikings, Saracens (who now occupied Spain), Basques and Saxons.
By the end of the millenium, France was divided into areas controlled by dukes and counts representing Gauls, Bretons, Danes, Aquitanians, Goths, Spanish and Gascons.