So I read a piece recently explaining why 40k's Imperium is so grimdark. Now don't get me wrong, I already knew the details, but i'd never seen it spelled out like this before. Thought you lot might like it - it makes for interesting reading. FYI, I DIDN'T WRITE THIS. The horrendous spelling, punctutation and grammar is not my fault. It just makes for awesome reading.
The Mechanicus does NOT have the technology. They haven't been living on some fancy paradise planet since pre-Fall. Mars is an anarchic nightmare shithole the moment you leave the safe zones into the kilometres of labyrinthine corridors beneath it full of rogue machinery, self-aware and malevolent AI from before the Fall, and the daemon programs of the Heresy. EVERYTHING in the databases is fucked. The databases are fragmented over the entire surface to the extent that it would be impossible to see one tenth of the total files in the ludicrously extended life of a Magos even assuming that they are completely safe to visit. And they are not.
The files have been corrupted into madness by the Fall, and the unleashing of the most potent informational warfare systems ever to exist to defeat the Iron Men. Nearly all of Mars was rendered uninhabitable, what they live in now is built on the top of the ruins. They send archeotech expeditions in to find shit, nearly all of them never come back. The sheer number of rogue war machine running around in there is sufficient to rape the mind. Then came the Heresy, which was not earth-exclusive. Mars as the second most critical planet in the Imperium was the site of fighting nearly as ferocious as on Terra, with Mechanicus loyalists and Hereteks fighting tooth, nail, and mechadendrite everywhere. Ancient machines were unleashed, viruses both normal and daemonic unleashed into all the computer systems. Nearly every single stored record on Mars was rendered unusable, and those that survived are half the time self-aware and don't like you, or daemonic and actively try to kill you.
If you come back with a schematic, it is almost certainly gibberish, and if it isn't, it's probably corrupted into uselessness. If it does come back whole it was probably malevolently fucked with so that instead of a Lasgun power cell it's a fucking grenade set to detonate the second you finish building it. Why do you think they want off-world STCs so damned much if they had them all here? The fucking Heresy is why. Off-world they only have to contend with the Fall's war and its effects on the machinery plus twenty thousand years of degradation with no maintenance. But at least off-world it'll probably just not work instead of actively seek to kill you.
Why do you think they seek to placate the Machine Spirit? It's because it exists. The fragments of trillions of self-aware programs, flourishing during the Dark Age of Technology and shattered by Man in his war with the Iron men, imprisoning the few who had not set themselves irrevocably into the machinery, a prison smashed wide open by the Heresy. Everything that can hold programming in the Imperium has a shard of a program in it. EVERYTHING. And you'd better fucking please it or it will do everything in its power to make your day shit. Sure, if it's a Lasgun it'll just not work or start shooting off rounds by itself, but if you piss off a Land Raider you can say bye-bye to half a continent. They apply these principles to things without spirits by habit, since they're so used to dealing with tanks that if not talked to just right might go rogue and annihilate the Manufactorum before they can be killed.
This is why they do not like ANYONE fucking with technology, because it is so rare to find anything that just works it is critical it not be compromised. That, and they do not have the actual knowledge to fuck with it intelligently, just through experimentation, which inevitably leads to slaughter. Pressing buttons to see what works is fine in a 21st century computer, but it is a very stupid thing to do at the helm of a 410th century starship with the destructive power to end solar systems. The entire knowledge base of humanity was lost. Not forgotten, but outright lost. Everything at all, poof. Nobody knows anything because the Fall fucked everything up and the Heresy double-fucked it. To rebuild the theoretical framework needed to design new technologies that don't kill everyone near them would require starting from the ground up. They don't have the time, and they never have.
This gets on to the point of war and what it does to technology. Someone will parrot that it makes it go much faster. Yes, it makes practical applications of technology go much faster. It also utterly stops all research on the scientific theories behind those technologies. This means that when war chugs along for a decade or two things get done. It means when it goes on too long you run out of theories to turn into technologies, and then you run out of technologies to apply. You stagnate. When you have been fighting in a war for survival in a drastically overextended empire, this is what happens. You are desperate for any extra materiel that can possibly be produced. Half your entire fucking military might went rogue, smashed the half that stayed, leaving you with the tattered shreds of a war machine to keep hold of an empire that was reaching straining point with an army far larger. There is no time for the sort of applied research programs that took Man twenty five thousand years to develop, in a time of unprecedented growth and prosperity.
This is also why the Adeptus Mechanicus insists on cargo cultism. It's because when you are dealing with things you barely understand because everything you knew about them was destroyed it is the safest and most reliable option. The rituals do not exists for mysticism, they exist because they are the most practical means of building, repairing and maintaining the equipment they have with the knowledge surviving. You don't understand why pressing that button makes it go, because the manual tried to take over your brain and the copies are all unreadable and the research base that would let you reverse-engineer it does not exist and cannot be built.
Why are the Tau doing so well with their technology? Because they had peace. Eight thousand years unmolested by any enemy and they were helped the entire time by the most advanced biological race in the galaxy. Give the Imperium eight thousand years of peace and I can guarantee you it will be harder than it was during the Great Crusade.