Smart AI Assistant

BlueBox AI offers a selection of on-premise Large Language Models (LLMs) that are ready to use right out of the box, including:
- Mistral 7B, Mixtral, MIstral Large
- LLAMA 3 7B, 13B, 70B,
- IBM watsonx.AI LLMs.

At the heart of the Bluebox AI mechanism lies the Elasticsearch NoSQL Vector DataWarehouse. This advanced data storage solution combines the flexibility of NoSQL databases with the powerful capabilities of vector search, enabling efficient handling of large datasets. Elasticsearch ensures fast and scalable data retrieval, making it an ideal choice for AI applications that require high performance and robust data management.

BlueBox AI serves as a robust backend system for a myriad of applications, including third party Chatbots, ERP apps, and BPM apps. The system boasts the capability to expose secure APIs, providing access to a range of microservices specifically designed for human-like chatting, text creation, summarization, and comparisons. This versatility allows seamless integration of BlueBox AI into diverse applications, enhancing their functionalities with advanced natural language processing capabilities.

Bluebox AI offers a widget that can be embedded on websites, seamlessly integrated with Livechat.com. This integration allows for enhanced user interaction and support, providing real-time assistance powered by AI.

The graphical user interface (GUI) stands as an independent component, open to modification, development, and adaptation according to the specific needs of each customer. Powered by React as its primary technology and utilizing Chakra UI components, the GUI provides a flexible and customizable interface that can be tailored to meet the unique requirements of diverse users.

BlueBox AI seamlessly links up with any data lake, a repository housing millions of unstructured file data, and conducts synchronization at predetermined intervals. The files undergo a comprehensive process, including reading, conversion, OCR (Optical Character Recognition), verification within permission contexts, and subsequent placement into BlueBox AI storage.

Leveraging sophisticated systems such as IBM watsonx.data, Presto, and Spark, BlueBox AI establishes integration with both structured and unstructured data lakes, ensuring a harmonious and efficient data management experience.

The system is meticulously scaled to accommodate a range of 100k-10 million documents, with storage capacity extending up to 10TB. Utilizing SSD discs and employing the fastest available mechanisms for input/output (IO) operations, the storage system is engineered to deliver optimal performance and efficiency.

In cases where local data proves insufficient for addressing complex queries, the system incorporates a component capable of accessing publicly available internet data or private data repositories provided by companies offering data access services. The search results obtained from these sources are seamlessly integrated with Total Search mechanisms, systematically ranked, and refined to deliver the most optimal answers for the given questions.

This specific component of the system is dedicated to OnPrem Large Language Model (LLM) processing, representing a pivotal aspect of its functionality. The efficiency of this processing unit varies in accordance with factors such as the type of LLM utilized, its specific configuration, and the extent of quantization applied. For instance, the system exhibits the capacity to concurrently serve 8 users for the more resource-intensive LLAMA2 70B, extending up to an impressive 120 concurrent users for the 3B LLMs. This adaptability ensures that the machine can effectively handle a spectrum of processing requirements, catering to diverse user needs and accommodating varying LLM specifications.

In the context of conducting similarity analysis within the Vector Database, a meticulous process involves configuring millions of document chunks for comparisons with the query. This demanding task necessitates substantial RAM memory and a high-speed CPU for optimal performance. To effectively support large-scale similarity analysis processes, a robust system with 3TB of RAM is deemed suitable, ensuring both the storage capacity and computational speed required for the intricate comparisons performed within the Vector DB.

2x 1.92TB NVMe SSDs
6x 7.68TB NVMe SSDs

1x 1G 4-port OCP Adapter – Management Connector

2x AMD EPYC 9634 processors, 84C, 2.25GHz

4x 2400W power supplies (230V)

4x NVIDIA ConnectX-7 200 GbE or NDR dual port