Note: This English version was generated by AI. The original article was written in German.
This proof of concept (PoC) was developed as part of the Special Week at Bern University of Applied Sciences. It shows how AI-powered features can simplify data updates in a CRM system to improve efficiency and usability. The PoC uses PyQt for GUI development and various AI APIs for its functionality.
Optimizing the CRM
In today’s data-driven business world, effective customer relationship management (CRM) is essential. But even the most advanced CRM systems are only as good as the data they contain. A project addressing data quality challenges in an existing CRM system yielded valuable insights.
Project Scope
This proof of concept (PoC) was developed by a project team as part of the Special Week at Bern University of Applied Sciences. PyQt was used to create a simple GUI that makes it possible to extract data from the web based on a list of URLs. The following information is collected through several steps:
- Company Analysis: A brief description of the company, what it does, and the industry it operates in.
- Competitor Analysis: 5–10 similar companies operating in the same industry are identified. A new query for the newly identified companies can also be launched directly.
- General Contact Details: Email address, phone number, address, and other relevant contact details.
- Contact People: Information about specific contact people relevant to the company, such as name, position, and email address. A screenshot of the website is taken to verify the accuracy of the collected data.
The Challenge: Inefficient Data and Manual Work
The initial problem was typical of many companies: The existing CRM system suffered from duplicate entries, outdated information, and incomplete records. These inconsistencies significantly undermined the effectiveness of sales and marketing efforts. Manual checks were time-consuming and tied up valuable resources that could be put to better use elsewhere. The goal was clear: The CRM data needed to be cleaned, validated, and enriched to improve its accuracy and relevance and increase the reliability of the system.
Two Approaches to Improving Data Quality
To meet these requirements, two primary approaches were developed: a robotic automation tool and an API & web scraping solution. Since the robotic automation solution was very complex and lacked flexibility, the focus shifted to the second solution.
API & Web Scraping Solution: In-Depth Analysis from the Web
The second solution uses publicly available APIs to collect, filter, and analyze data from external sources. The workflow consists of several stages:
Step by Step toward Comprehensive Information:
- URL Processing (Scraping): URLs from a cleaned input file are passed to a scraping API. It extracts all links from the homepage, filters relevant URLs, and saves them. This approach makes it easy to integrate filtering mechanisms.
- URL Analysis (Intelligent API Use): The Perplexity API (an LLM with internet access) is used to collect information about companies, domains, and similar companies. This greatly simplifies data collection and provides up-to-date information as well as summaries of the current state of the pages.
- Contact Data Scraping: Another API is used to extract precise and comprehensive contact details from the filtered pages. All collected contact details are consolidated into a single file per domain. To check their accuracy, a screenshot of the page is also captured, allowing quick verification without opening a browser.
- Data Visualization: After each processing step, the collected data is visualized in a user-friendly graphical user interface (GUI) to provide a quick overview.
Outlook and Future Improvements
The project holds enormous potential for future extensions:
- Robotic Automation: Integration of future AI APIs with web browsing capabilities and automatic UI adaptation through image recognition or machine learning.
- API & Scraping: Expansion of data sources (e.g., industry-specific databases), parallel processing and caching to increase speed and reduce costs, improved data validation, and optimization of AI models. A more sophisticated GUI with interactive data visualizations could further improve the user experience.
Conclusion: A Strategic Asset for the Future
Implementing these solutions can transform a company’s CRM system into a more reliable and strategic asset. By focusing on automation and data analysis, this project lays the foundation for a powerful CRM and market analysis tool.