How to Run Gemma 4 on Your Phone: A Hands-On Guide

Most AI tools rely on the internet, sending your prompts to remote servers for processing before returning results. This process has always been invisible to users. Google changes that with Gemma 4! Which if configured properly, runs directly on your phone, eliminating the need for constant connectivity.

With a one-time download, everything runs locally on your device, keeping your data private. You can access it through Google AI Edge Gallery App. In this article, we explore how to use the app and what you can build with it without Internet, once it has bee configured locally on your device.

What Exactly is Gemma 4?

The Gemma 4 family consists of four distinct models, each optimized for various hardware requirements by Google. The E2B version is a low-resource device, while the E4B version has been designed for higher throughput. The larger models are truly impressive; for example, the 31B dense model ranks #3 in terms of all open-source models worldwide, while the 26B MoE model sits at #5, outperforming many larger models. 

While these benchmarks are noteworthy, there are many other reasons to appreciate this new generation of artificial intelligence (AI). The entire Gemma 4 family has been engineered to provide capabilities beyond simple chat; it will be able to perform complex logic and facilitate agentic workflows, process word, video, and audio, and use more than 140 different languages. 

For devices such as phones, the two edge variants of Gemma 4 (E2B and E4B) have been created specifically for low-resource hardware. These models can handle vision, audio, and text data; include function calls; and be small enough to fit within the storage limitations of mobile platforms.

Read more: Google’s Gemma 4: Hands-On 

The App that Makes it Possible

Google has released their AI Edge Gallery application which works on both Android and iOS platforms. Your smartphone performs all processing tasks without needing any cloud service. The application functions as an open-source software.  

The following features of AI Edge Gallery make it essential for our use case: 

• AI Chat with Thinking Mode: The model demonstrates its reasoning process by explaining your question through its complete reasoning path 
• Ask Image: You can use your camera to scan any object which you want to investigate and ask questions regarding it. 
• Audio Scribe: The tool allows users to convert spoken audio into text or different languages without needing an internet connection. 
• Agent Skills: The system can perform multiple tasks without human guidance by utilizing resources such as Wikipedia. 
• Prompt Lab: Users can evaluate their prompts by controlling the temperature settings to improve their results. 

The Agent Skills feature stands out as an essential element of the system. It marks one of the earliest instances where consumers can use multi-step agentic AI technology which operates entirely offline on their mobile devices. 

Why this Actually Matters?

The ability to run AI on local systems provides multiple benefits which go beyond its aesthetic appeal. The three main advantages of this technology present authentic benefits to users: 

• The model operates entirely on your device which establishes privacy as the primary requirement. The application does not transmit any of your shared content which includes prompts and responses and images to Google or any other server. The system operates through its network connection which only requires the model to be downloaded.  
• No connectivity needed. The system functions properly when you are on a flight or in a basement or in an area with weak signal. Gemma 4 operates independently of your current location. The software provides complete functionality after you download it. 
• The model becomes freely usable for an indefinite period after you complete its download. It operates without using any background resources because it requires no tokens or credits or subscriptions.  

The licensing agreement establishes another requirement. Google released Gemma 4 under an Apache 2.0 license which permits businesses to use and modify and build on the models without any usage restrictions.

Which Model Should you Pick?

Most people become confused at this point. The size of a model does not determine its value because larger models do not always outperform smaller ones. The four variants of Gemma 4 include Effective 2B (E2B) and Effective 4B (E4B) and 26B Mixture of Experts and 31B Dense. For phones, you need to use the E2B and E4B systems according to Business Today. 

The following provides an essential overview: 

• Gemma 4 E2B requires less than 1.5GB RAM to operate. The system provides immediate responses to simple inquiries while generating brief summaries through its Q&A. 
• Gemma 4 E4B requires approximately 2.5GB RAM for its operation. The system can execute more advanced visual assignments through its enhanced reasoning abilities and its improved function calling system. 

The E2B system performs better than other systems for basic operations that show high-performance needs. The E4B system offers better performance than other systems because it handles complex function schemas and multiple function options better than other systems.  

You should begin with E2B as your starting point. Switch to E4B when you observe that it fails to handle multi-step reasoning tasks. 

Getting Started with Gemma 4

Step 1: Go to the Google Play Store (for Android) or Apple Store (for iOS), type in Google AI Edge Gallery and download the app.

Step 2: Open the app. You will be brought to the main menu and see all five modes that you can choose from (AI Chat, Ask an Image, Audio Scribe, Agent Skills, and Prompt Lab). 

Step 3: Navigate to the Model Management section and download either Gemma 4 E2B or Gemma 4 E4B. The only time you need to be connected to the internet is when downloading these models; you only must do this once. 

Step 4: After downloading, you can turn on airplane mode. From this point on, all functions will work without being connected to the internet.

Task1: Building a Sudoku Game using AI Chat feature

Here, we’ll be developing the sudoku game using Gemma 4 on Google AI Edge Gallery by selecting the AI Chat feature: 

1. Start by opening the app and selecting AI Chat, then enable Thinking Mode. 
2. Type “Please create a sudoku game using Html Css Javascript in order to have a timer, check solution functions, and ensure that it is mobile-friendly” (no quotes). 
3. The model will perform its logic before producing complete code. 
4. When done, simply copy all the code and paste it into a new text file that will be saved as an .html file, then open the new html file using any web browser; your game should now be working. 

Note: If you want to have more cleanly constructed code from the outset, try using Gemma 4 E4B. Also, should issues arise with functions that have previously worked correctly, simply tell Gemma which function you need trouble with and ask her for help repairing it. 

Review Analysis:

When I prompted E2B model then it just stopped mid-task but after prompting the E4B model, it produced the output. The model gave us html code file with thorough instruction which was quite helpful in case of non-tech users. Though, it could have also shown us a frontend interface which was a little disappointing. Also, since it’s running in offline mode, it’s taking alot of time which shows us the limitation of the model. 

Task2: Automate Tasks with Agent Skills

1. Enable the Map, Email, and Wikipedia skills by tapping Agent Skills and enable them. 
2. After that, you can test the agent by giving them the following three requests one after the other: 
   • “Find a coffee shop that is closest to me and place it on a map for me.” 
   • “Compose an email for me to send to John indicating that I’m going to be 10 minutes late and send it.” 
3. After each request, the agent will break the request down into individual tasks, call the appropriate tool(s), and confirm with you before completing and sending any work. 

Note: You can track precisely which skills were used by the agent after each step. The agent is completely transparent in their actions with you as well. 

Review Analysis:

Results were somewhat varied for multiple agent skill types. For the first query, Map generally provided results where the location looked correct on the map, but it should have been able to detect my location on its own instead of explicitly asking me.  

For the second query, it loaded the skill for ‘send-email’ appropriately. After the execution of skill, it showed that message has been sent but it didn’t have any info where it sent the message, which is like a huge drawback. The response time and occasional breakdowns of the ability to complete the task demonstrated that there is still a significant amount of improvement to make within the Use of Agentic AI Type Devices. 

What Can’t it Do (Yet)?

When we talk about Gemma 4, it has some limitations as well: 

• There is a battery drain associated with using inference, as it requires significantly more computer power than other types of apps. Therefore, will deplete your battery much faster than other types of apps. Devices with dedicated NPUs are much better at managing inference battery usage than using cpu-only for inference. For example, a Pixel 9 Pro with a Gemma 4 4B will deplete its battery at a much slower rate than using only a pure CPU. 
• In addition, the larger versions of Gemma (26B and 31B) cannot be used on a mobile phone; you need a laptop with a lot of RAMS to run them. The E2B and E4B devices are fine for daily tasks; however, they will not be replacing frontier cloud models anytime soon. 
• Finally, the edge models have a 128k context window, which is good; however, since phones do not have as much memory as computers do, you will need to keep your conversations relatively short on a phone. 

Conclusion 

The term “AI on your phone” throughout multiple years described as a basic interface which accessed remote cloud APIs. The system processed your information through a circuitous route which passed through an unprotected server.  

Gemma 4 establishes an entirely new connection between two different entities. 

Your current pocket device can perform three functions which include transcribing talks and analysing visual content and solving difficult challenges through offline capabilities. Previously, system operation required a complete server facility. Now it requires an app download. 

The era of AI running silently on your pocket device, with no server involved, is no longer a research demo.  

Frequently Asked Questions

Data Science Trainee at Analytics Vidhya
I am currently working as a Data Science Trainee at Analytics Vidhya, where I focus on building data-driven solutions and applying AI/ML techniques to solve real-world business problems. My work allows me to explore advanced analytics, machine learning, and AI applications that empower organizations to make smarter, evidence-based decisions.
With a strong foundation in computer science, software development, and data analytics, I am passionate about leveraging AI to create impactful, scalable solutions that bridge the gap between technology and business.
📩 You can also reach out to me at [email protected]