AI Text Classification Web App

We are in need of a skilled web app developer to create an AI-powered text classification system. We are looking for a developer to make a web app that is able to accept any kind of raw text or a pdf and extract raw text from it. So, the initial page can have 2 options for Data Source: Raw Text and PDF (We might add support for other input formats as well in the future). Next, once we have the text available, it should split the text based on the lines. It can split on the full stop character. And then every line must be sequentially given to a ML model. This would be a text classification huggingface model, which can be instantiated via this code:

from transformers import pipeline

pipe = pipeline("text-classification", model="legal-text-classification/IndianLegalBERT")

Feel free to explore the above model yourself. It will give you 15 labels with different scores. If the score of the top scoring label is greater than 0.8, then it must highlight that line on the webpage with the color associated with that label. And the model must be fed with this line along with the next line and in case the score is again greater than 0.8 for the same label, it must highlight both the lines with the color associated with that label as done earlier. However, in case this time, the label is different but score is still greater than 0.8, then we must highlight both the lines/all the lines fed into the model for which this label was received with the color associated with this label. In case the model doesn't give more than 0.8 for any label for a particular input, we move forward to the next line. And all the older lines need not be sent to the model again. It starts with this new line itself.

For example, let's say there are 5 sentences separated by full stops: S1,S2,S3,S4,S5 and 3 labels: L1,L2,L3 having 3 associated colors: C1,C2,C3. Now let's say the model gave L1 as the label for S1 with a score of 0.84. Now, since the score is more than 0.8, it must highlight the S1 with C1. Moving forward, we push S1 appended with S2 to the model. Lets say the model predicted L3 as the label for this combination with a score of 0.82. Since the score is greater than 0.8, both the sentences must be highlighted with C3. It is critical that the color of S1 also changes since the combination gave a different label. Now since the last prediction's score was greater than 0.8, we append S3 to the last input (S1+S2) and feed into the model. Let's assume it gave L2 with a score of 0.95. This means the color of all 3 sentences must change to C2. Previously, S1 and S2 were highlighted with C3, now it must be changed to C2. Now, the 4 sentences would be fed into the model and let's say its highest scoring label was L3 but with a score of 0.78. Since this doesn't meet the threshold of 0.8, we neither highlight S4 with any color nor change colors of the 3 previous sentences. Since the last prediction didn't meet the classification, we don't append the next sentence (S5) to the last input (S1+S2+S3+S4). Instead we only feed S5 to the model. Let's say it gave L1 with a score of 0.92. So, we highlight the sentence S5 with C1. Since there are no more sentences left, the process ends right here. It is also important to note although we show a legend to the user showing which label each color represents, the color-label combination must remain consistent among all the users. We do not need dynamic color mapping. But the legend must show only those colors that are highlighted which means only those colors would be present in the legend whose associated labels were predicted but not altered. A basic functional frontend GUI will suffice, nothing fancy is required but the integration with backend must work perfectly. Feel free to use any tech stack you are comfortable with. We are only concerned with the output which must be fully functional and must have a reasonable performance.

We must write a code to solve this in a modular fashion. The user will only input the data once and it is our responsibility to highlight the predicted classes using the method explained above. It is essential that this particular method is followed accurately without any flaws. Feel free to ask any doubts or clarifications that you might have regarding any corner cases before starting to write the code. We can discuss another example with a higher number of sentences as well to gain a better understanding about how the flow should work if required. For example, let's consider 8 sentences, 3 labels and 3 colours (1 color associated with each label):

- S1: L1 (0.85) - Highlight S1 with C1.

- S1 + S2: L3 (0.92) - Highlight S1 and S2 with C3 (changing the color of S1).

- S1 + S2 + S3: L2 (0.75) - No highlighting, as the score is below the threshold.

- S4: L1 (0.90) - Highlight S4 with C1.

- S4 + S5: L1 (0.88) - Highlight S4 and S5 with C1.

- S4 + S5 + S6: L2 (0.83) - Highlight S4, S5, and S6 with C2 (changing the color of S4 and S5).

- S4 + S5 + S6 + S7: L3 (0.65) - No highlighting.

- S8: L3 (0.91) - Highlight S8 with C3.

Python JavaScript Flask Node.js AI Model Integration

Project ID: #38069574

About the project