How to introduce AI technologies into a medium-sized office (May 2024)

Abstract

Here we present a plan to introduce AI technologies into a typical IT office with an overview of technologies available as of May 2024. A wise approach would suggest starting with a limited pilot project first.

Hypothetical scenario

In an era of advancing AI technologies, companies are met with a change in how to introduce AI-enabled technologies and tools into regular work processes. Typical dilemmas companies have are: what technologies are available, what hardware they need, and where to start.

The question is of course, will AI tools deliver the promise, i.e. will they be useful as it is promised? Since licenses and hardware are expensive, seems prudent to evaluate tools first.

In this text, we discuss a hypothetical introduction of AI technologies into some medium-sized 100-person office/company, which is focused mainly on IT-related services/development. We are talking from the point of view of a moment in time being May 2024 and news and technologies available. We assume the hypothetical company in question has no previous experience with AI tools but has a curiosity and interest in seeing if such technologies can be useful. The hypothetical office we have in mind is typically based on the x86/Windows platform and various development tools.

Plan with two directions/focuses

I would divide the plan into 2 directions.

The main direction, focused on the Microsoft AI initiative, 85% of the effort
The auxiliary direction, focused on alternative technologies, 15% of the effort

The main direction focused on the Microsoft AI initiative

Microsoft has put a lot of effort into thinking about where and how in the modern office workplace and developing environment AI technologies can be applied and in which way they can be helpful. It is not just Microsoft because of Microsoft, they did a really good job thinking and planning over a longer period of time. The result is a set of hardware and software products that are planned and produced, not all finished and released at this moment in time.

The main direction was set by the Microsoft event on May 20th, 2024, with the proposed CopilotPC hardware standard and the Copilot family of products. These are BIG, BIG changes that are setting a path for the future in personal computing. They basically want every PC to have a powerful Neural network processor (NPU) and are writing products for it. I know of no other company at the moment that is so bravely putting a vision for both hardware and software and making ready-to-use AI products in such an impressive portfolio.

So, Microsoft is saying “Stay with us, we have a vision for AI and AI products for you”. And they sound serious to me. What is the catch? Microsoft wants to make money from its hard work and licenses are expensive. For example, for office products, Copilot for Microsoft 365 is $30 per user per month. For an office of 100 employees, that is $36,000 per year just for software licenses to have smart AI-enabled Office products. If you want AI-enabled products, you pay for separate licenses again. And is no longer buy a product and use it forever, the new trick is they sell a license “per user per month”.

Still, because of their clear vision and a big portfolio of products, if you want to go AI direction, the Microsoft CopilotPC initiative is the way to go.

But, stay grounded regarding expectations of AI-enabled products. Try before you buy, because marketing is very strong and they tend to overpromise on the capabilities of AI-enabled tools. Even supercomputer-based ChatGPT-40 was found to “hallucinate” 3% of the time, so is not really “production ready” and requires output to be checked/proofread by the humans. Be careful, many AI companies, including Microsoft, just want your money and will sell you anything under the marketing name AI-enabled.

Also, most of the Copilot and other AI-enabled Microsoft products are still in development and new to the market, so maybe it is wise to wait a bit for them to mature and get bugs-free. Taking early versions, you might find yourself in the position of practically real-world testing discovering and reporting unknown bugs.

The introduction of Microsoft AI technologies can be divided into 3 elements.

Acquiring CopilotPC hardware
Acquiring AI-enabled software. That software can be divided into 2 groups
- Software focused on AI-Cloud services: It can be introduced without introducing CopilotPC hardware first.
- Software focused on CopilotPC hardware: You need to buy CopilotPC hardware first to run this software.
Acquiring training tools: videos and books/manuals (PDFs)

The auxiliary direction focused on alternative technologies

Here I would put an effort into seeing/monitoring the usability of other technologies, at the moment of writing, I find personally interesting technologies.

ChatGPT-4o by OpenAI
Meta Llama 3 OpenSource AI

The reason I put ChatGPT-4o by OpenAI into the auxiliary group is, that while the tool is very entertaining, I do not see it very easily integrated into the work process and other tools. Maybe I am wrong on this one, but if I need to generate an email in ChatGPT and then copy-paste it every time, it is not going to be very efficient. That is why Microsoft Copilot tools with integration are supposed to be much more efficient. Time will tell us what is the best.

I work in an IT-developing environment, and professionally made, license-free, tools like Meta Llama 3 Open Source AI are always interesting. Microsoft is planning on providing Small Language Model (SML) Phi Silica, specifically designed for new CopilotPC hardware. That way developers will have the ability to develop custom AI systems on the CopilotPC platform. Currently, Meta Llama 3 requires powerful Linux hardware but is license-free, and let’s just keep an eye on it.

Buying new CopilotPC-compatible hardware

In its vision, Microsoft is proposing a new hardware platform: CopilotPC. There are 2 versions, CopilotPC/x86, and CopilotPC/ARM.

CopilotPC/ARM hardware/laptops will be available as of June 2024.
CopilotPC/x86 hardware/laptops will be available later. With Intel chips expected to be finished in Q3 2024, laptops will be available probably December 2024.

What is your local IT department saying?

If you are eager to catch the AI wave and have a strong local IT department, go for CopilotPC/ARM right now. Windows 11 is compatible with that platform, as are many Microsoft applications. Adobe is in the process of migrating its products to CopilotPC/ARM/Windows.

I work in a 100+ people office that is mostly x86/Windows based and the IT department is quite busy supporting and managing all the user's problems and their hardware/laptops.

Typical IT departments consist of engineers of various experience and skill levels. Employees tend to fluctuate, so you can not base your enterprise technology base on one person. Typically, IT engineers have good experience with x86-64 hardware platforms and relevant hardware and admin tools. If you push them to a new ARM laptop platform, many will feel dislike in their work. Even worse, despite claimed compatibilities, some of their tools might not work on new hardware. The problem can be serious and cause big delays if for example their favorite USB-booting tool is no longer working on the ARM-laptops platform, or if the licensed disk-imaging tools they used for years do not work in the new environment.

If you can wait until December 2024 with the technology introduction, maybe is worth playing safe and waiting for the availability of CopilotPC/x86 hardware. You will make life much easier for your local IT department and all x86 tools will still work. You can still in the meantime introduce those AI tools that require only AI-Cloud services.

Used AI-technology company levels

You need to define to which level you want AI technology introduced into the company. I see several levels/objectives that can be reached with the available tools and services on the market.

Objecitve1: Usage of commercial productivity AI tools in regular office work. So, depending on your office/company business domain, you want to start using AI-enabled tools like Microsoft Office 365 Copilot, or some of the new AI-enabled Adobe productivity tools. Again, they can be divided into 2 groups, those that are exclusively based on AI-Cloud services and those that require CopilotPC hardware, like most new versions of Adobe products.
Objective 2: Usage of AI tools for software development. If you are a software developer shop, you will definitely be thrilled with the idea of enhancing productivity with AI tools. Here I talk about using tools like Microsoft GitHub Copilot or ChatGPT to generate snippets of code.
Objective 3: Creating AI-enabled applications. Sooner or later, developers will start producing AI-enabled applications, and for that, they need some tools and smarter hardware. Microsoft has announced the availability of Phi-Silica, which is a Small Language Model (SML), specifically designed for the Neural Processing Units (NPUs) in new Copilot+PCs. Also, Meta Llama3 is an Open Source of Large Language Model LLM for the Linux platform.

Start with Pilot projects

It is always good to start with Pilot projects. That will put less stress on the IT department if they need to prepare new hardware for a limited number of users first and gain the necessary experience with installing and maintaining the new platforms like CopilotPC/x86/Windows.

The IT department also needs to get some experience with Copilot products and licenses. Also, licenses are expensive and you want to see if the product's AI-enhancements are useful or maybe we need to wait a couple of years until they mature.

I would divide Pilot projects into 2 categories.

Pilot-AI-Cloud: Here I mean testing AI tools that are purely AI-cloud services based.
Pilot-AI-CopilotPC: Here I mean testing tools that require CopilotPC hardware.

Pilot-AI-Cloud

In this pilot project, I mean testing AI tools that are purely AI-Cloud services based and you need no particular hardware for them. You can start this pilot project right away. Depending on your business domain, you can test AI tools of your interest. The key thing is to use new tools in real work processes, to get feedback on how useful they are in the real world. Some ideas might be:

Get 2 licenses for GitHub Copilot (different versions are $10, $19, and $39 per month) and give them to your Senior Software Engineers to test them in their everyday work. After 2 months you will have in-house experience with AI tools and an opinion about the usefulness of such tools.
Get 2 licenses for ChatCPT-4o Plus ($20 per month) and give them to two employees, and at least one Senior Software Engineer. The problem is only that the Chat GPT Desktop app for Windows users will be available later this year. After 2 months you will have in-house experience are they useful in generating code and in other office work?
Get 2 licenses for Microsoft Copilot for Microsoft 365 ($30 per month per user). Give them to 2 employees and you will get their opinion on how useful they are.

Pilot-AI-CopilotPC

In this pilot project, I mean testing tools that require CopilotPC hardware. Some ideas might be.

You buy 2 laptops, be it Copilot/ARM/Windows or CopilotPC/x86/Windows with licenses, and give them to 2 Senior Employees. You need to migrate completely their work environment to the new laptops so that AI tools get used in real-world situations. I see no point in just “playing” with new laptops. They need to be used in day-to-day work.
You get some CopilotPC-enabled software like Adobe products to test them in real work scenarios.
You get some CopilotPC-enabled software like Microsoft products (I am not sure which is not Cloud-based at the time of writing) and test them in real-world scenarios.

Conclusion

The presented plan is the opinion of one software engineer and AI enthusiast and is more kind of a set of ideas to inspire and provide some advice to anyone interested. Each company/office is specific and will have its own take on the problem. Take anything you like from this plan, and add your own ideas, to create your company’s custom plan.