Case Study IV of IV– Benchmarking PatSeer’s Semantic search relevance

The table lists all the elements from the paragraph, followed by the patent claim numbers where these elements appeared. The subsequent column shows the exact text from the patent claims that matched the respective key elements. For instance, “concentrated salt water” was matched with “concentrated sea water.” Similarly, the key components of an osmotic membrane component, like reverse osmosis and forward osmosis membrane components, were appropriately matched and, an assisting device was matched with an assisting component. The fourth column assesses the degree of similarity between the paragraph and the claims of the patent document. The last column of the table indicates the extent of overlap between each element and the corresponding Title, Abstract, and Summary of Invention. This alignment was determined through manual examination.

It is evident that all the key elements mentioned in the paragraph were precisely covered in the Title, Abstract, Claims, and Summary of Invention of the first record identified through the AI Semantic search.

A similar evaluation was conducted for the remaining set of records, and we have compiled all the results into a single table provided below.

To calculate the final semantic performance score for the top 10 results, a scoring system was employed: assigning 1 point for full overlap, 0.5 point for partial overlap, and 0 point for no overlap and positional weightage was given based on the position of the result in top 10. Based on the calculations done the final semantic performance score for the top 10 results in this case was 82.27%.