AI governance in German companies: what I keep seeing, and what it costs

If I had to explain AI governance in one sentence, it would be this: AI governance is how a company decides which AI it uses, for what, under whose responsibility and with which controls.

A lot of German companies have not made those decisions yet. They use AI every day, in official projects, in Copilot licences, in the AI features that arrived with the last CRM update, and in private chatbot accounts. What they lack is an overview and a way to decide.

AI use in German companies has moved fast. In a Bitkom survey from September 2025, 36% of companies with 20 or more employees used AI, almost double the year before. The same survey names legal uncertainty as the top barrier, cited by 53% of companies.

Employees are not waiting for their employers. In Bitkom's October 2025 survey, 40% of companies knew or suspected that staff use private AI tools at work. Only 26% provided an official generative AI tool, and only 23% had AI usage guidelines. Another Bitkom survey found that only one in five employees had received AI training at work.

Regulation adds pressure. Parts of the EU AI Act already apply: prohibited practices and the AI literacy duty since February 2025, and transparency rules for chatbots and AI-generated content since August 2026. The obligations for high-risk systems, such as AI used in hiring or credit decisions, were moved to December 2027 (see Article 1, see Article 2). In Germany, the Bundestag passed the national implementation law in June 2026, which makes the Bundesnetzagentur the central AI supervisor.

So AI is spreading inside companies faster than anyone is tracking it, while expectations from regulators, customers and employees are rising.

AI governance sounds large. In practice it comes down to a handful of decisions, made once and kept up to date:

1. Inventory: Which AI systems and AI features we use, including those inside third-party software.

2. Purpose and risk: What each system is used for, and how much harm a wrong output could cause.

3. Ownership: One accountable person per system, and one role that coordinates across the company.

4. Data rules: Which data each system may use, and where that data goes.

5. Lifecycle: How we test, monitor and review a system when the model, the data or the use case changes.

With these in place, a new use case gets a fast yes or a clear no. Compliance with the AI Act, GDPR and works council agreements becomes a result of the process, rather than a separate project each time.

I work on AI architecture and implementation projects, and the same patterns come up again and again:

1. There is no AI inventory. The answer to the question: "Where do you use AI today?"covers the official projects. It misses the AI inside everyday software and the tools teams bought on a credit card.

2. Nobody owns AI decisions. Legal thinks it is an IT topic. IT thinks it is a compliance topic. The business assumes someone has already checked it.

3. Governance exists only on paper. There is a policy, but no approved tool, no named contact and no process that people actually use.

4. The approval board becomes the bottleneck. Every use case, low risk or high risk, goes through the same manual review, and teams give up or work around it.

5. Vendor claims are accepted without evidence. "Our tool is compliant" ends up on a slide, without documentation of what the system does, what data it uses and how it is tested.

6. Pilots cannot go live. A working prototype stalls for months because data protection, works council and security questions only come up at the end.

• Treating AI governance as a legal topic only. Lawyers are essential, but the decisions are about business purpose, architecture and data. Governance works when business, IT, legal, data protection and, in Germany, often the works council decide together.

• Banning AI instead of offering an approved tool. A ban pushes usage into private accounts, where you have no control over data at all.

• Reviewing everything the same way. A meeting-summary tool and an AI that pre-screens job applicants do not need the same process. Without risk tiers, governance slows down the harmless cases and still misses the critical ones.

• Leaving vendor AI out of scope. Companies that deploy AI systems have their own duties under the AI Act, for example on human oversight and transparency towards the people affected.

• Waiting for regulation to force the work. Logging, human oversight and documentation are architecture decisions. Adding them to a finished system later costs far more than designing them in.

The AI Act fines get the headlines: up to EUR 35 million or 7% of global annual turnover for prohibited practices, and up to EUR 15 million or 3% for most other breaches. In my experience the business consequences arrive earlier:

• AI adoption slows down. Each new use case becomes a fresh debate, and competitors with clear rules move ahead.

• Pilot budgets are wasted. Prototypes that cannot pass approval never deliver value.

• Data leaks. Customer data and trade secrets end up in tools outside your control.

• Decisions cannot be defended. When an AI-supported decision goes wrong, you need to show who decided what and on which basis.

• Deals are lost. Large customers and public buyers increasingly ask suppliers how they govern AI. Without an answer, you fail the questionnaire.

As my approach to AI is more grounded in reality than hype, here's where to start without setting up a large programme:

1. Build the AI inventory. List every AI system and AI feature in use, with owner, purpose and data used.

2. Set risk tiers. Define two or three levels with a matching approval path, so low-risk tools get a fast lane.

3. Name owners. One accountable person per system, and one central role for AI governance.

4. Offer an approved tool and a short usage policy. Two pages with clear rules beat thirty pages nobody reads.

5. Make AI literacy role-based. Developers, HR, sales and management need different training.

6. Ask vendors for evidence. Documentation, data handling, testing, and support for your own obligations.

7. Build governance into new projects from day one. Logging, oversight and documentation cost little at the start.

The companies I see moving fastest with AI are the ones that settled these questions early. Once the rules are clear, teams stop asking for permission on every use case and start building.

I am an independent consultant with no ties to any AI vendor or model provider. If you want a second pair of eyes on your AI governance setup, send me a message.