The supply chain due diligence act is a German law designed to improve the protection of human rights and the environment in global supply chains and was introduced at the beginning of this year.
As part of the supply chain due diligence act, it is required to gain detailed information about all suppliers to assess their risk of non-compliance with the law. This information includes various data points such as locations, owners, responsible key people, industries in which companies are operating, revenue, number of employees, availability of certificates, and more. However, this information is often available in an unstructured format on the websites of the companies, necessitating further processing.
The objective of this project is to design, implement, and deploy an AI-powered webcrawler that efficiently navigates through business websites in a hierarchical manner, identifies relevant data points, and employs NLP techniques to extract crucial information related to the company's operations and structure. The webcrawler should be capable of handling various types of websites and deliver accurate and comprehensive results. End results is supposed to be the information described in the introduction.
a. AI-powered Webcrawler Prototype: Develop a functional webcrawler that effectively scans websites and utilizes NLP technologies to extract the desired information (industry, ISO certificates, top management, sublocations, products, services, customers, and subfirms).
b. Data Processing and Visualization: Implement a data processing pipeline that cleans, organizes, and presents the extracted data in a user-friendly format for further analysis and visualization.
c. NLP Processing of the Crawled Data: Utilize advanced NLP techniques to process the crawled data and structure it into a pre-processed format suitable for subsequent steps.
d. Neural Network-based Categorization: Employ neural networks to execute categorizations and enhance the accuracy and efficiency of information classification.
This project demands a diverse set of skills, including: