- According to Imperva’s 2026 Bad Bot Report, more than half of all web traffic is now automated. Your website is read by software as much as by people.
- AI assistants and AI agents increasingly search, compare and act on behalf of your clients. If they cannot read your site, they recommend someone else.
- An agent-ready website is not a gadget: server-rendered HTML, clean structure, structured data, llms.txt, a deliberate robots.txt policy, speed and clear facts.
- This is how 123 Consultancy builds websites, starting with our own site, 123.tn. Write to website@123.tn to discuss yours.
For twenty years, a company website had one audience: people, usually arriving from Google. That is no longer true. Today a large part of the visits to your site come from software: search engine crawlers, AI crawlers that feed language models, AI assistants answering a question in real time, and now AI agents that browse, compare and fill in forms on behalf of a person. Some of this traffic is useful, some is not, but all of it changes the same question: can a machine understand what you do, what it costs and how to contact you?
In this article we explain why a website matters even more in the AI era, what the numbers say, and the concrete practices we use at 123 Consultancy to build websites that people enjoy and that AI agents can reach, read and use.
The web’s second audience: what the numbers say
Several independent reports published in 2025 and 2026 describe the same trend. They measure different things on different networks, so the figures are not directly comparable, but the direction is clear.
- Bots now outnumber humans. According to the Imperva (Thales) 2026 Bad Bot Report, automated traffic accounted for more than 53% of all web traffic in 2025, up from 51% the year before. About 40% of all traffic came from malicious bots.
- AI traffic is growing fast. According to HUMAN Security’s 2026 State of AI Traffic report, AI-driven traffic grew 187% over 2025, and traffic from AI agents and agentic browsers grew by several thousand percent year over year (HUMAN reports +7,851%), from a very small base.
- AI crawlers are already a visible share of page requests. According to the Cloudflare Radar 2025 Year in Review, AI bots (excluding Googlebot) made up about 4.2% of HTML page requests on average in 2025, and Googlebot alone about 4.5%.

Not all bots are welcome: scrapers and attack bots must be filtered. But the useful ones, search engines, AI assistants and the agents working for your future clients, decide more and more often whether your company is found, quoted and contacted.
Why your website matters even more in the AI era
A common reaction is: “If people ask ChatGPT, Claude, Gemini or Perplexity instead of visiting websites, why invest in a website?” The answer is that these assistants still need a source. When someone asks “which agency builds multilingual websites in Tunis?” or “how much does a showcase website cost in Tunisia?”, the assistant searches the web, reads pages and builds an answer from what it understood. Your website is the source it reads.
- It is the reference about your company. Social media posts and directory listings are fragmented and controlled by others. Your website is the only place where your services, prices, references and contact details are complete, current and under your control.
- AI answers quote websites. AI search engines and assistants cite the pages they used. A clear, well-structured page has a better chance of being read correctly and cited.
- AI agents act, not only read. An AI agent can be asked to “find three suppliers, compare their offers and request a quote”. If your contact form is unusable by software, or your offer is hidden in an image, the agent moves on to a competitor.
- Trust still happens on your site. After an AI recommendation, people still check the company’s website before calling. A slow or outdated site loses them at the last step.
What an AI agent actually “sees” on your site
A person sees colours, photos and layout. An AI crawler or agent mostly sees the code behind it: the HTML text, the headings, the links, the forms and the structured data. If the important information only exists inside images, animations or content loaded late by JavaScript, many bots simply do not see it.

That is why we design every website on three layers at once: a visual layer for people, a clean HTML structure that any software can parse, and a data layer that states the key facts explicitly.
How 123 Consultancy builds websites that AI agents can reach, read and use
These practices are part of every website we build. Most of them cost little when they are planned from the start, and a lot when they are added afterwards.
1. Content rendered on the server
The text of each page is present in the HTML sent by the server, not assembled later in the browser. Many crawlers and AI assistants do not run JavaScript, or run it only partially; server-rendered pages are readable by all of them, and they load faster for people too.
2. A clean, semantic structure
One h1 per page, logical h2 and h3 headings, real lists and tables, descriptive links, images with alternative text, and forms whose fields have proper labels. This is what lets an agent understand a page and, if needed, fill in a contact or quote form correctly.
3. Structured data (schema.org JSON-LD)
We describe the essential facts in a machine-readable format: the Organization (name, logo, contact details), the Services or Products you offer, the FAQ, the BreadcrumbList navigation, and articles on the blog. Search engines and AI systems use this data to understand who you are and what you offer without guessing.
4. An llms.txt file
llms.txt is a simple text file at the root of the site that summarizes, in plain language, who you are, what you offer and which pages matter most, with links. It is a proposed convention rather than an official standard, and not every AI system uses it yet, but it costs almost nothing and gives language models a clean entry point. Our own site has one at 123.tn/llms.txt.
5. A deliberate robots.txt policy for AI crawlers
The robots.txt file tells crawlers what they may visit. AI crawlers identify themselves with names such as GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot (Perplexity) or the Google-Extended token (use of your content for Google’s AI models). There is no single right answer: some companies want maximum visibility in AI answers, others want to keep their content out of model training. What matters is that the choice is made consciously, with you, instead of being left to defaults. On 123.tn we allow AI crawlers, because we want our services to be found and quoted.
6. An XML sitemap and hreflang for every language
A complete sitemap.xml lists every page, its last update and its images, so crawlers find new content quickly. For multilingual sites, hreflang tags link each page to its translations, so French speakers get the French page and English speakers the English one, in Google and in AI answers.
7. Fast pages
Light images, minimal scripts, good caching and a reliable host. Speed matters for people on mobile networks, for Google’s ranking, and for bots that give up on slow pages.
8. Clear facts that an assistant can quote
The questions your clients ask an AI assistant are simple: what do you do, for whom, where, at what price, how long does it take, how do I contact you? We make sure the answers are written plainly on the site, in text, and grouped in an FAQ. Vague marketing slogans are hard to quote; clear facts are easy.
Our own website, 123.tn, applies all of these practices: server-rendered pages in English and French, JSON-LD structured data, an llms.txt file, an XML sitemap with hreflang alternates, and an explicit robots.txt.
Agent-ready website checklist
| Check | Why it matters for AI agents |
|---|---|
| Main text present in the HTML source | Bots that do not run JavaScript can still read it |
| One h1 and logical headings | The topic and sections of the page are unambiguous |
| Labelled form fields | An agent can fill in a contact or quote request |
| JSON-LD: Organization, Service, FAQ, BreadcrumbList | Key facts are stated explicitly, not guessed |
| llms.txt at the root | A plain-language summary with the important links |
| robots.txt with a decided AI crawler policy | You choose who may read and reuse your content |
| sitemap.xml and hreflang | All pages and all languages are discovered |
| Fast loading on mobile | Neither people nor bots give up on the page |
| Prices, services, contact and FAQ in plain text | Assistants can quote accurate answers |
What about price?
Making a website readable by AI agents is mostly a matter of method, not of extra features: when it is part of the project from the start, the impact on the budget is small. The main factors that drive the price of a website remain the number of pages and languages, custom design, features and content. We explain them in detail in our guide to website creation in Tunisia: costs, steps and mistakes to avoid.
Contact 123 Consultancy
Do you need a new website, or want to know whether your current site is readable by AI assistants and agents? Write to us at website@123.tn with a few lines about your company, your goals and your current site if you have one. We will reply with concrete recommendations and, if useful, a clear proposal. You can also see our websites and videos offer or use our contact form.
Websites by 123 Consultancy
A website your clients love, and AI agents understand.
Fast, multilingual, well structured and ready for AI search. Tell us about your project and we reply within 24 hours.
FAQ
Does a company still need a website if people use AI assistants?
Yes. AI assistants and AI search engines build their answers from sources they read on the web. Your website is the most complete and reliable source about your company, and it is where people check you before they call.
What is an agent-ready website?
A website that AI crawlers, assistants and agents can reach, read and use: content rendered on the server, clean HTML structure, labelled forms, schema.org structured data, an llms.txt file, a deliberate robots.txt policy, a sitemap, fast pages and clear facts such as services, prices and contact details.
What is llms.txt?
A plain-text file placed at the root of a website that summarizes the company and lists its most important pages for language models. It is a proposed convention, not an official standard, and not all AI systems use it yet, but it is cheap to add.
Should I block AI crawlers like GPTBot or ClaudeBot?
It depends on your goals. Allowing them increases your chances of being found and quoted in AI answers; blocking some of them can keep your content out of model training. The decision should be deliberate and written in your robots.txt.
Does this make a website more expensive?
Not much, if it is planned from the start. Most of the work is method: server-rendered pages, clean structure and structured data. Adding it to an existing site is also possible and usually starts with an audit.
How do I contact 123 Consultancy about a website project?
Email website@123.tn with a short description of your company and goals, or use the contact form on 123.tn. We reply within 24 hours.
