Enabling AI-Based Food Scanning with a Mobile App
FOOD SERVICE | MOBILE DEVELOPMENT | AI, CLOUD COMPUTING
A popular YouTube food blogger turned to Amitech to develop a scalable and performant solution for end users to analyze grocery products based on their ingredients and make informed purchasing decisions.
The customer
The customer is a US-based food blogger and producer/distributor of organic goods. With over 5+ million social media followers, the influencer creates recipes and provides shopping advice, promoting healthy eating habits.
The need
The customer runs a YouTube blog with over 3 million followers, providing food reviews in pursuit of the balance between product ingredients and cost. To digitize the process and expand reach, the blogger decided to deliver a mobile scanner for buyers to overview a product’s nutrient profile right at the store and make informed purchasing decisions.
Partnering with Amitech, the customer wanted to develop a mobile app with an extensive product catalog that would scan products, automatically analyze ingredients, and provide shopping advice.
The challenges
Since the system had to store arrays of product data, scalability, high availability, performance, and fault tolerance were a top priority.
Initially, the customer planned to enrich the product base by manually entering data. However, the process was time consuming and error-prone, preventing accurate ingredient analysis.
The solution
After revising the requirements, the engineers at Amitech delivered a system consisting of a back end to store data and a cross-platform mobile client for managers and buyers.
Using the app, end users can scan a product’s barcode and get a comprehensive ingredient overview coupled with shopping advice. Both end users and managers can enrich data by adding images, product descriptions, and ingredients. Managers also oversee and approve submitted items, regulate user access, look into statistics, etc.
Built as a set of microservices and hosted on AWS, the solution ensures scalability, high availability, fault tolerance, and security. The microservices are responsible for catalog and credentials storage, user authentication, and e-mail distribution.
The solution's user interface
To automate manual data entry, the engineers integrated two nutrition systems: FoodData Central and Nutritionix. The developers employed data mapping to ensure proper data retrieval and transmission from various sources. Employing persistent data management, the team guaranteed data accuracy, consistency, and integrity over time. Then, the experts at Amitech delivered a mechanism that analyzes ingredient data and provides recommendations.
The engineers also implemented AWS Textract and ChatGPT for image recognition and natural language processing (NLP). Now buyers can take a photo of a desired product, and the app will extract data from labels, facilitating the generation of product descriptions and ingredient information.
To enable convenient search for specific items, the developers integrated Elasticsearch. The solution also leverages the React Native Reanimated library for smooth and interactive animations within the mobile app, enhancing the overall user experience.
The outcome
Together with Amitech, the customer delivered a mobile app for scanning and analyzing grocery products, representing a fully fledged mobile product information management system.
2+ million requests and 100K+ scans daily
with the ability to elastically scale during increased loads
AI-based image recognition and NLP
to quickly capture and process product label data, eliminating manual effort and reducing errors
A product catalog with 1 million items
optimized to enable a comprehensive, updated, and accurate data flow
Highlights
An efficient food scanner for buyers to analyze product labels and get shopping advice
A mobile PIM to grant managers uninterrupted access to relevant product data
Scalability, high availability, and fault tolerance
Reduced manual effort with AI-based image recognition and natural language processing
Improved performance thanks to data optimizations
Integrations with government-regulated databases for accurate product catalog
Technology Stack
Platforms
Amazon Web Services
Programming languages
TypeScript, Java
Frameworks and tools
Node.js, React Native, Amazon Textract, ChatGPT, Amazon S3, AWS Lambda, AWS CloudFront, Amazon EC2, Elastic Load Balancer, Amazon ECS, Amazon Route 53, Amazon ECR, REST API, Spring, Redux, Firebase
Database
PostgreSQL
