The Model Is the Easy Part: Architecting Document Ingestion for a Custom Azure AI Application

When organizations begin planning a custom artificial intelligence application, the first conversation usually centers on the model.

Should the solution use Azure OpenAI? Which model deployment should be selected? Does the application need an AI agent? Should it run in Azure App Service, Azure Container Apps, or Kubernetes?

Those are legitimate architecture decisions, but they are rarely the hardest part of the project.

How will the application securely ingest, process, govern, and continuously update the organization's existing documents?

For most established businesses, the information that gives an AI solution real value is not stored in a clean cloud-native database. It is distributed across Windows file servers, SMB shares, network-attached storage, departmental folders, SharePoint libraries, scanned records, technical manuals, policies, engineering documents, contracts, invoices, and years of accumulated business data.

Deploying a large language model is relatively straightforward. Building a reliable document-ingestion platform around it is where the real engineering begins.

An AI Model Does Not Automatically Know Your Business

Azure OpenAI provides powerful language models, but it does not independently crawl an on-premises file server, monitor an SMB share, preserve NTFS permissions, extract text from scanned PDFs, generate search indexes, or determine whether a document has been replaced or deleted.

Those responsibilities belong to the surrounding application architecture.

A production-grade enterprise AI solution normally needs to:

1. Connect securely to the source repository.
2. Discover eligible files and capture their metadata.
3. Detect additions, modifications, renames, and deletions.
4. Transfer documents into a controlled Azure landing zone.
5. Validate and, where required, scan the files.
6. Extract text, layout, tables, and structured data.
7. Divide the content into useful retrieval chunks.
8. Generate vector embeddings.
9. Build and maintain a searchable index.
10. Enforce document-level authorization during retrieval.
11. Supply only relevant content to the language model.
12. Monitor failures, retries, latency, cost, and data freshness.

Without those capabilities, the result is not an enterprise knowledge platform. It is a chatbot with a manual upload feature.

The Reference Architecture

A common hybrid architecture for ingesting on-premises documents into a custom Azure AI application separates data movement, processing, retrieval, model inference, hosting, and security. Those responsibilities should not be collapsed into one oversized service.

Step 1: Connect the On-Premises File Server to Azure

The first design decision is how documents will leave the local network.

For scheduled or batch-oriented ingestion, Azure Data Factory with a self-hosted integration runtime is often the most supportable enterprise option. The integration runtime is installed on a server inside the private network and gives Azure Data Factory access to local data sources, including file-system paths and network shares.

A typical flow is:

\\FILESERVER\DepartmentShare
        ->
Self-Hosted Integration Runtime
        ->
Azure Data Factory Copy Activity
        ->
Azure Blob Storage

This model provides centralized pipeline orchestration, scheduling, monitoring, retries, parameterization, and incremental copy logic. It also avoids directly exposing SMB to the public internet. The self-hosted integration runtime uses outbound HTTP-based connectivity, which is easier to place behind an existing firewall than an inbound path to the file server.

Azure Data Factory is especially useful when the organization needs to synchronize multiple departmental shares, run ingestion during defined maintenance windows, copy files based on modification dates or folder rules, maintain separate development and production pipelines, track failures centrally, and scale ingestion without embedding file-transfer logic into the AI application.

When a Custom Ingestion Agent Is Better

Azure Data Factory is not the only valid approach.

A custom Windows service or local agent may be more appropriate when the solution requires near-real-time change detection, advanced file filtering, custom hashing, specialized metadata collection, or direct interaction with NTFS access-control lists.

A purpose-built agent can use FileSystemWatcher or scheduled scans, calculate cryptographic hashes, capture the original UNC path and file owner, read NTFS access-control entries, upload files through the Azure Storage SDK, publish a processing message to Azure Service Bus, and maintain a local checkpoint if the Azure connection is unavailable.

The agent should normally initiate outbound TLS connections to Azure rather than requiring Azure to reach into the local network.

Containerized Ingestion Agents

A containerized ingestion agent can be useful when the organization wants standardized deployment, version control, and repeatable configuration across multiple sites.

The container could run on an on-premises container host, an edge platform, or a managed Kubernetes environment. Its job should remain narrowly defined: discover files, package metadata, transmit the content securely, and report status.

Do not deploy Kubernetes merely because the workload involves AI. For a single ingestion worker, a Windows service or standard container runtime may be operationally cleaner.

Step 2: Build a Controlled Azure Landing Zone

Documents should not be sent directly from the file server to the language model.

They should first land in Azure Blob Storage or Azure Data Lake Storage Gen2, where the pipeline can maintain a durable and auditable source copy.

A practical storage layout might include:

/raw
/incoming
/processing
/processed
/extracted
/quarantine
/failed
/deleted
/manifests

The landing zone creates resiliency. If Document Intelligence is unavailable, an Azure Function fails, or the search index is being rebuilt, the original document remains available for reprocessing.

Each uploaded file should be accompanied by metadata such as original UNC path, source server and share, file name and extension, created and modified timestamps, file size and hash, source department, file owner, security classification, CUI or export-control markings, NTFS or application permissions, ingestion timestamp, processing version, and indexing status.

This metadata becomes critical when the application needs to cite a source, filter by department, remove an obsolete file, or prove where an answer originated.

Step 3: Make the Pipeline Event-Driven

Once a document reaches Azure Storage, the pipeline should begin processing it without relying on constant polling.

Azure Event Grid can publish events when blobs are created, replaced, or deleted. Those events can be delivered to Azure Functions, Logic Apps, webhooks, queues, or other subscribers.

A common pattern is:

BlobCreated event
        ->
Azure Event Grid
        ->
Azure Function
        ->
Validation and processing workflow

An event-driven architecture reduces unnecessary storage scans and allows processing to begin shortly after a document arrives.

For higher-volume or business-critical environments, place Azure Service Bus or Azure Queue Storage between the event source and the processing workers. The queue provides buffering, retry handling, dead-lettering, and protection against sudden ingestion spikes.

A 50,000-document migration should not overwhelm the same processing service used for daily incremental updates.

Step 4: Use Azure Functions for Orchestration

Azure Functions is well suited for short-lived, event-driven activities.

A document-ingestion function might validate the file extension and content type, confirm that the file is within scope, verify the file hash, check for duplicate or superseded versions, write a processing record to a database, submit the document to Azure Document Intelligence, publish work to a queue, update status metadata, route failures to a quarantine container, and send telemetry to Application Insights.

For multistep or long-running workflows, Durable Functions can coordinate asynchronous operations and maintain state between stages.

A durable orchestration could register the document, submit it for extraction, wait for the extraction result, run content normalization, generate chunks and embeddings, update the search index, mark the document as available, and notify an administrator if any stage fails.

Functions should be orchestration components, not monolithic document-processing engines. CPU-intensive conversions, large-document parsing, and specialized libraries may be better placed in a containerized worker.

Step 5: Extract Meaning From the Documents

Documents are not uniform.

A digitally generated policy PDF is fundamentally different from a scanned maintenance form, an invoice, a table-heavy engineering document, or a handwritten inspection record.

Azure Document Intelligence in Foundry Tools can extract text, tables, key-value pairs, selection marks, layout, and structured fields from documents. It includes prebuilt and custom models for different document-processing scenarios.

The extraction strategy should be selected based on the file type:

• Use native text extraction when the document already contains reliable digital text.
• Use OCR and layout analysis for scanned PDFs and images.
• Use prebuilt models for common forms such as invoices and receipts.
• Use custom extraction models for organization-specific forms.
• Use classification when different document categories require different workflows.
• Preserve page numbers, headings, tables, and coordinates when citations or visual references matter.

The most expensive AI service is not automatically the best choice for every document. A mature pipeline routes files to the least complex service capable of producing the required output.

Step 6: Normalize, Chunk, and Enrich the Content

A retrieval system should not treat a 600-page manual as a single search record.

The extracted text must be divided into logical sections, commonly called chunks. Each chunk should be small enough for accurate retrieval but large enough to retain meaning.

Good chunking preserves document title, chapter and section headings, paragraph relationships, page numbers, table boundaries, revision identifiers, effective dates, source paths, and security metadata.

Poor chunking can split a warning from the procedure it governs, separate a table header from its rows, or return a technically correct sentence without the surrounding limitation.

Chunking can be based on token count, paragraph boundaries, heading hierarchy, page layout, semantic similarity, document type, or table and figure boundaries.

The processing pipeline may also enrich each chunk with summaries, keywords, named entities, contract numbers, equipment identifiers, department labels, document classifications, suggested questions, and normalized acronyms.

Azure OpenAI can assist with semantic enrichment, but deterministic information such as the original path, file permissions, revision date, and file hash should come from the source system, not from model inference.

Step 7: Create the Azure AI Search Index

Azure AI Search is commonly used as the retrieval layer for a custom Azure AI solution.

It supports full-text search, vector search, semantic ranking, hybrid search, metadata filters, facets, integrated vectorization, and indexes over structured and unstructured content.

Hybrid search combines keyword and vector queries in the same request and merges their results. This is often more reliable than using vector similarity alone because it can balance semantic meaning with exact terminology, product numbers, contract identifiers, acronyms, and technical phrases.

For example, a user might ask:

What is the inspection interval for the hydraulic pump installed on platform XJ-417?

Vector search helps identify semantically related maintenance content. Keyword search helps preserve the exact equipment identifier. Metadata filters can restrict the results to the correct program, revision, classification, and user authorization.

The index should generally contain chunk text, vector embedding, document ID, parent document title, source URI or UNC path, page or section reference, revision and effective date, security principals or access groups, department and classification, processing timestamp, and deletion or supersession status.

Step 8: Use Retrieval-Augmented Generation

Once the documents are indexed, the application can implement retrieval-augmented generation, commonly called RAG.

The runtime flow is:

User question
    ->
Authentication and authorization
    ->
Query rewriting or embedding
    ->
Azure AI Search hybrid query
    ->
Security-trimmed document chunks
    ->
Azure OpenAI prompt
    ->
Grounded response with citations

The model is not being permanently trained on every document. Instead, the application retrieves the most relevant authorized content at runtime and supplies that content to the model as context.

This approach provides several operational advantages: new documents can become available without retraining a model, deleted or superseded content can be removed from the index, responses can cite the original source, search results can be filtered by the current user, different model deployments can use the same knowledge index, and retrieval can be improved independently from the user interface.

The model is the reasoning layer. Azure AI Search is the retrieval layer. The ingestion pipeline is what keeps the retrieval layer trustworthy.

Where Should the Application Components Run?

Azure provides several compute platforms. Selecting the right one depends on workload behavior, scaling requirements, deployment model, and operational maturity.

Azure App Service

Azure App Service is often the best choice for a conventional web application or REST API.

Use it when the solution needs a stable web application, Microsoft Entra ID authentication, straightforward CI/CD, deployment slots, managed domains and certificates, VNet integration, and minimal infrastructure management.

For many custom AI portals, App Service provides the simplest path from development to production.

Azure Container Apps

Azure Container Apps is a strong fit for containerized APIs, background workers, event consumers, and independently scalable microservices.

Use it when the architecture separates components such as an AI orchestration API, document-processing worker, embedding worker, file-conversion service, queue consumer, scheduled ingestion job, agent service, or prompt-management API.

Container Apps supports revisions, autoscaling, managed identities, internal environments, and private networking patterns without requiring the organization to operate Kubernetes directly.

A useful design might host the user-facing web application in App Service while running document processors and queue workers in Container Apps.

Azure Functions

Use Azure Functions for event-driven orchestration and relatively short processing operations.

Functions are ideal for reacting to storage events, publishing messages, updating status records, and connecting managed Azure services. They are less appropriate for every workload simply because they are serverless.

Azure Container Instances

Azure Container Instances can run isolated or short-lived containers without a broader orchestration platform. It may be appropriate for an occasional batch conversion job or specialized utility, but it is generally not the primary hosting platform for a complete enterprise AI application.

Azure Kubernetes Service

Azure Kubernetes Service provides maximum orchestration control, but it also introduces a significant operational burden.

Use AKS when the application genuinely requires Kubernetes capabilities such as complex service meshes, extensive portability requirements, custom operators, large-scale multi-service orchestration, or specialized node pools.

Do not introduce AKS solely because the application uses artificial intelligence. Architectural complexity should be justified by workload requirements.

Security Must Extend to the Document Level

A custom AI application must not become a new path around existing access controls.

Suppose the Human Resources director can read a disciplinary-records folder, but an engineer cannot. If both users can access the AI application, the retrieval layer must still prevent the engineer from receiving HR content.

Application authentication alone is not enough.

The architecture should account for Microsoft Entra ID authentication, managed identities, Azure role-based access control, NTFS ACL ingestion, group and user security principals, SharePoint or repository permissions, document classifications, sensitivity labels, CUI boundaries, export-controlled information, data-retention rules, legal holds, and audit logging.

A common approach is to translate source permissions into searchable security metadata. The application resolves the current user's Entra ID group memberships and applies corresponding filters to every Azure AI Search query.

Security trimming must occur before retrieved content is sent to Azure OpenAI.

Use Private Connectivity and Eliminate Embedded Secrets

A production architecture should minimize public exposure and static credentials.

Depending on the environment, the design may use site-to-site VPN, Azure ExpressRoute, private endpoints, VNet integration, Private DNS zones, storage firewalls, Azure Container Apps internal environments, managed identities, Azure Key Vault, customer-managed keys, Microsoft Defender for Cloud, and Azure Policy.

Managed identities allow App Service, Functions, and Container Apps to authenticate to supported Azure resources without storing service-account passwords or connection strings in application code.

The objective is not merely to encrypt the file transfer. The complete data path, from the on-premises source to storage, processing, indexing, retrieval, and model inference, must be included in the security design.

Initial Migration and Continuous Synchronization Are Different Workloads

A proof of concept may begin with 100 manually uploaded PDFs.

A production rollout may involve millions of files, multiple terabytes of data, duplicate documents, inconsistent naming conventions, inaccessible folders, unsupported formats, and years of stale content.

The initial migration should be treated as a controlled bulk-ingestion project. Daily operations should use incremental synchronization.

The pipeline needs to handle new, modified, renamed, moved, and deleted files, along with permission changes, superseded revisions, duplicates, unsupported formats, password-protected documents, and processing failures.

File hashes, stable document IDs, checkpoints, manifests, and source-system change data are essential. Relying only on file names will eventually create duplicate records or stale index entries.

Deletion Is Part of the Architecture

Many ingestion designs focus exclusively on adding documents. That is not enough.

If a document is deleted, its chunks and vectors must also be removed from Azure AI Search. If a policy is replaced, the old version should be marked as superseded or removed according to retention requirements. If a user loses access to a folder, the search security metadata must be updated.

An AI application that continues citing deleted or unauthorized documents is not trustworthy.

The pipeline should define a deletion workflow for detecting the source change, identifying the stable document ID, removing or updating index records, retaining required audit evidence, invalidating stale caches, and recording the action in operational logs.

Monitoring and Observability Are Production Requirements

A document-ingestion pipeline is a distributed system. Failures will occur.

Operational teams need visibility into files discovered and transferred, processing duration, extraction failures, queue depth, Function errors, container restarts, search indexing failures, documents awaiting reprocessing, time from source change to search availability, token and model consumption, storage growth, search latency, and retrieval quality.

Azure Monitor, Log Analytics, and Application Insights should provide a correlated view across the ingestion agent, Data Factory, Functions, Container Apps, Document Intelligence, Azure AI Search, Azure OpenAI, and the application itself.

That is a measurable production outcome. "The chatbot works during the demonstration" is not.

A Practical Azure Deployment Pattern

For many midmarket and enterprise organizations, the following architecture provides a strong starting point.

• On-premises layer: Windows file servers or NAS, SMB departmental shares, Azure Data Factory self-hosted integration runtime, optional custom ingestion agent, and outbound-only encrypted connectivity.
• Azure data layer: Azure Blob Storage or Data Lake Storage Gen2, raw, processed, quarantine, and failed containers, metadata database or manifest store, private endpoints, and storage firewall.
• Processing layer: Azure Event Grid, Azure Service Bus or Storage Queues, Azure Functions or Durable Functions, Azure Container Apps workers, and Azure Document Intelligence.
• Retrieval and AI layer: Azure OpenAI embedding deployment, Azure AI Search hybrid and vector index, Azure OpenAI language-model deployment, prompt orchestration, and citation logic.
• Application layer: Azure App Service or Azure Container Apps, Microsoft Entra ID authentication, role and group-based authorization, and Application Insights.
• Governance and security layer: Managed identities, Azure Key Vault, Private Link, Azure Monitor, Log Analytics, Microsoft Defender for Cloud, Azure Policy, Cost Management, backup, and disaster recovery.

This architecture is modular. Each service has a defined responsibility and can be scaled, secured, tested, and replaced independently.

Common Design Mistakes

Uploading Documents Directly to the Model

A language model is not a document-management system. Files need durable storage, indexing, lifecycle handling, and authorization.

Treating the Initial Upload as the Ingestion Strategy

A one-time import creates a static demonstration. Production requires continuous synchronization.

Ignoring Permissions Until the End

Security metadata must be designed into the ingestion and index schemas from the beginning.

Using Vector Search Alone

Exact identifiers, acronyms, part numbers, and policy references often benefit from keyword and hybrid retrieval.

Running Every Task in One Web Application

Long-running extraction and indexing jobs should not compete with interactive user requests.

Selecting AKS by Default

Kubernetes is powerful, but it is not free operationally. App Service and Container Apps are often the better platform decisions.

Failing to Track Deletions and Revisions

A stale answer cited from an obsolete policy can be more damaging than no answer at all.

Measuring Model Quality Without Measuring Retrieval

Many apparent model failures are actually ingestion, chunking, metadata, or search-ranking failures.

The Bottom Line

The quality of a custom AI application depends on far more than the language model.

The real platform includes the hybrid connection, ingestion agent, storage landing zone, event architecture, processing workers, extraction service, chunking strategy, embedding model, search index, authorization model, application runtime, and monitoring and governance controls.

Organizations that design document ingestion as a first-class architectural capability can build AI applications that remain current, secure, explainable, and operationally supportable.

Organizations that treat document ingestion as a manual upload step usually end up with an impressive proof of concept and an unreliable production system.

Before selecting the model, design the data path.

Frequently Asked Questions

How Fletcher Technology Group Can Help

Fletcher Technology Group designs and implements private, enterprise-grade AI solutions that connect organizational data to Azure AI services without sacrificing security, governance, or operational control.

Our work includes hybrid document-ingestion architecture, Azure OpenAI integration, Azure AI Search, custom RAG applications, Microsoft Entra ID authentication, private networking, and deployment into customer-controlled Azure environments.