Siraj Ibn Wahhaj, 40, was arrested along with four other people on Friday at the desert compound in Amalia, New Mexico, and has been charged with 11 counts of child abuse.
Eleven children aged one to 15 were found living on the compound in filthy conditions, prosecutors said, with no clean water or electricity. Authorities also found little in the way of food.
Wahhaj is also under investigation for the death of a 12th child - possibly his son, Abdul-Ghani Wahhaj - whose body was discovered on the property.
Prosecutors quoted the foster parent of one of the children as claiming that Wahhaj "had trained the child in the use of an assault rifle in preparation for future school shootings."
As a result, he faces another investigation for teaching "the use, application or making of any firearm, destructive device or technique capable of causing injury or death" with the intention that the knowledge be used unlawfully - which is classed as a felony.
"The Defendant ... is under investigation ... based upon the training of children with weapons in furtherance of a conspiracy to commit school shootings," prosecutors said Wednesday in a court filing requesting that Wahhaj be held in custody.
"Trip hazards, woods with nails sticking up, broken glass, bottles, and open trenches littered the property," it added.
"Trash scattered around the property, no clean water, no electricity, no indoor plumbing and the children were clothed in rags."
Prosecutors also said loaded firearms were found at the compound.
Three women believed to be the mothers of the 11 children found at the compound were also arrested in Friday's raid along with another man, Lucas Morten, and charged with child abuse. Wahhaj is wanted in the state of Georgia for the alleged abduction of his four-year-old son.