Introduction

Preface

I started writing this paper to help my own organization work through the challenges of adopting generative AI (GenAI). We needed to understand what AI is, how it can be useful and where it is not, and what our options were for gaining access to it. We needed answers on security and governance, on the benefits and challenges of running AI on-premises, and on what an air-gapped deployment would look like and require. We needed to know the financial costs and risks, the challenges of workforce adoption and training, the technologies required to deploy locally, and the facility considerations for running GenAI-capable servers on site. We also needed an honest comparison between cloud AI services and open-weight models and open-source tools. Behind all of that sat a host of other questions, some we knew to ask and many we did not.

The result is an introduction to the information a small or medium size organization needs to build out its own on-premises AI infrastructure. It is written for no particular team and no single purpose. It does lean toward engineering, scientific, and general knowledge work, and it includes examples of how AI can benefit those domains.

I come to this as a senior computer engineer and data systems architect. For more than ten years I’ve built large, distributed, multi-threaded, real-time data systems in modern C++ on Enterprise Linux. I also led the technical teams that deliver them, including systems that process and analyze real-time data across many machines. I’m an NVIDIA Certified Associate in AI Infrastructure and Operations with the qualifications to design, deploy, and operate on-premises AI on NVIDIA accelerators. That work runs from model selection, serving, and fine-tuning to air-gapped deployment, retrieval-augmented generation, and agent tooling. It also covers the data stores, security, and capacity planning that make those systems dependable in practice.

I wrote this paper over several months, between April and October 2026. I worked on it in the bits of free time left over between my full-time job, friends, family responsibilities, housework, and the rest of life. In those spare moments I would think through talking points and the direction of the paper. I then worked with Anthropic’s Claude AI models to refine those ideas and fold them into the paper where they fit.

I worked with the models the way a film director works with a crew. I set the scope, the arguments, and the standard each section had to meet. The models did the crew’s work: building the research registry, drafting and revising sections, checking edits against the house style, converting citations, and helping build the website. I reviewed every scene and kept final cut. I checked the sources, edited the work, and decided what was published, and I’m responsible for every claim in it. AI is one more tool in that work. When people who know how to use it apply it responsibly, effectively, and ethically, it makes their work faster and helps them produce higher-quality results.

That back-and-forth also taught me a good deal about what works, and what does not, when writing a large document with AI. This kind of document depends on validated technical information organized for several audiences at once. The workflow that carried this paper rests on four pieces: a research registry built before drafting, a written style guide, a drafting order set by the dependencies between sections, and a verification pass that checks every citation against its live source before a section is published.

If one piece of advice runs through this paper, it is to start small. Buy what today’s proven workloads justify, build skills and evidence alongside the hardware, and add capacity as the case for it grows. I believe organizations that start this way and keep building will see a large payoff over the next 5 to 10 years. I hope you enjoy reading this paper, learn a thing or two, and are able to make good use of the information contained within.

Organization Size Definition

Organization size carries no single official definition in the United States, so this paper fixes its terms at the outset. The Small Business Administration (SBA) Office of Advocacy treats a small business as an independent firm with fewer than 500 employees for research purposes, a threshold that covers 99.9 percent of US firms and 45.9 percent of private sector employment [1]. The regulatory size standards governing federal programs work differently, being industry specific and expressed in either number of employees or average annual receipts but never both for the same industry, so for many professional, scientific, and technical services classifications the applicable measure is revenue and no federal headcount threshold applies [2]. The Census Bureau, tabulating by enterprise, defines very small enterprises as those with fewer than 20 employees, small enterprises as 20 to 99, medium enterprises as 100 to 499, and large enterprises as 500 or more, noting that these terms are not equivalent to the SBA’s [3]. International Data Corporation (IDC), segmenting the technology market rather than the economy as a whole, applies the same medium band of 100 to 499 employees, places small business at 10 to 99, and carries its small and midsize business category through a further 500 to 999 tier before reaching very large organizations above 1,000 employees [4]. Because those two independently derived schemes converge, this paper treats small as fewer than 100 employees and medium as 100 to 499, uses the phrase small to medium organization for the combined range below 500, and notes explicitly wherever a recommendation continues to hold up to 999 employees. Readers working to European conventions should note the lower ceiling there, where a small and medium-sized enterprise (SME) employs fewer than 250 persons subject to turnover and balance sheet tests [5], supplemented since 2025 by a small mid-cap category covering enterprises below 750 employees [6].