PRHLT PrIx search engine.
Guidelines for the IBEM Mathematical Expression Search in Images
- Overview
- Examples of queries
- Confidence level, hierarchy, etc.
- Viewing search results
- Starting a new search
- Advanced searching
Overview
This interface allows users to search a collection of more than 11,000
pages from 1,098 scientific documents containing mathematical expressions.
The collection has been automatically processed from document images,
without requiring access to the original source files or manual transcription
of the mathematical content.
The resulting search engine enables efficient search of mathematical
expressions and sub-expressions directly from document images. Users
can perform exact or partial-expression searches and inspect the
corresponding occurrences within the original document pages.
The PRHLT research center has processed these documents using advanced
mathematical expression recognition and probabilistic indexing technologies
(PrIx). Rather than relying on a single transcription for each expression,
PrIx preserves recognition uncertainty and indexes multiple competing
hypotheses, improving the recoverability of mathematical content affected
by recognition ambiguities.
One of the main scientific objectives of this work was to investigate how
recognition uncertainty can be preserved and exploited for indexing and
search of mathematical expressions in document image collections. To this
end, the project combines mathematical expression recognition, uncertainty-aware
indexing, structure-aware segmentation of expressions and sub-expressions,
and efficient matching techniques that enable search directly over document
images without requiring manually verified transcriptions.
This demonstrator is the result of research conducted at the
PRHLT research center
of the Universitat Politècnica de València on mathematical
expression recognition, probabilistic indexing, and search in document
image collections. The technologies presented here build upon research
developed within the
IBEM,
DeepPattern,
SimancasSearch,
and
EUDEEP projects.
This work was partially supported by Generalitat Valenciana under
predoctoral grant CIACIF/2021/313, by grant PID2020-116813RB-I00
funded by MCIN/AEI/10.13039/501100011033, and by grant PID2024-161104OB-C21
funded by MICIU/AEI/10.13039/501100011033 and by the European Regional
Development Fund (ERDF/FEDER), European Union.
Happy searching!
Examples of simple and advanced queries are given first,
followed by a detailed explanation of all the search modes
and operators used in the examples.
Searching and Viewing Search Results
Mathematical queries can be entered directly using standard
LaTeX notation or generated automatically from uploaded or
cropped images of mathematical expressions. When an image-based input
method is used, the recognized
LaTeX expression is
automatically inserted into the search box, where it can be inspected
and edited before searching.
Before executing the search, the interface displays both a rendered
visualization of the query and, when applicable, its normalized
representation. These previews allow the query to be verified before
searching.
After selecting the desired search options, pressing the
Search button executes the query within the current
browsing context. Searches are contextual: when initiated from the home
page, the query is performed over the entire collection; when a document
is open, the search is restricted to that document; and when viewing an
individual page, only that page is searched. Returning to a higher level
of the hierarchy broadens the search scope accordingly.
Collection-Level Results
Collection-wide searches return the list of matching collections. Each
collection entry displays a representative thumbnail together with the
number of documents containing one or more matches. Selecting a collection
opens the corresponding document-level results.
Document-Level Results
Document-level results list all documents within the selected collection that
contain matches. Each document entry displays a thumbnail of the document together
with the number of pages with associated matching expressions. Selecting a
document opens the corresponding page-level results.
Page-Level Results
The page-level view displays page images together with the number
of matching expressions found on that page. Selecting a
page allows inspection of the matching expression within the original
document image.
Page Context
The page context view displays the original document page with the selected
mathematical expression highlighted. The colour of the box surrounding the
matched expressions indicates the confidence level, with green being the
highest and red the lowest.
Navigation controls allow users
to browse between multiple matches on the same page while preserving
the surrounding textual and mathematical context.
Starting a New Search
A new search can be started at any time by entering a new query or
modifying the current one. Searches are always performed within the
current browsing context. Consequently, if a document or an individual
page is currently open, the search is restricted to that document or
page, respectively.
To perform a collection-wide search, return to the home page before
executing the query. Clicking Home in the navigation
path preserves the current query while broadening the search scope to
the collection level. Clicking IBEM Mathematical Expression
Search in Images in the upper-left corner also returns to
the home page, but additionally clears the current query and restores
the default search interface.
Searches can likewise be focused on a particular document by first
opening that document from the collection-level results. Subsequent
queries will then be evaluated only within the selected document until
the search scope is changed again.
Search Controls
The interface provides several controls that modify how queries are
processed and matched.
The Normalize option converts the query into the canonical
representation used by the search engine before matching. This option is
recommended for most searches, as it allows users to write familiar
LaTeX expressions without following the internal indexing format.
The Segment option automatically decomposes the normalized
query into structurally meaningful sub-expressions. Mathematical expressions
are interpreted as hierarchical structures, where delimiters (such as
{, }, {@, and }@) define
nested branches, while relational, binary, and punctuation operators provide
additional segmentation points within these nesting scopes. The resulting
sub-expressions are searched independently and combined during matching. When
enabled, Min.Tokens specifies the minimum number of tokens
required for a generated segment, while Segm.Coverage defines
the minimum percentage of query segments that must be matched. Lower coverage
values allow more partial matches, whereas higher values require a larger
fraction of the query structure to be present in the matched expression.
The Confidence control is provided as both a numeric input
and a slider. It specifies the minimum confidence score required for a match
to be returned. Higher confidence thresholds produce fewer but generally more
reliable matches, whereas lower thresholds allow additional lower-scoring
recognition hypotheses to be considered, increasing recall at the expense of
potentially introducing less reliable matches.
The Max.Results control specifies the maximum number of
matches displayed for a query.
Expression Search
The following examples illustrate searches for mathematical expressions using
standard LaTeX notation. Depending on the query and search settings,
matches may correspond to complete expressions or to structurally meaningful
parts of larger expressions.
- a^2+b^2=c^2 (Options: Normalize, Confidence=50)
- E=mc^2 (Options: Normalize, Confidence=50)
- \sqrt{x^2+y^2} (Options: Normalize, Confidence=50)
- \frac{n(n+1)}{2} (Options: Normalize, Confidence=50)
- \frac{n(n-1)}{2} (Options: Normalize, Confidence=50)
- \Omega_M=\frac{1}{2}\Omega_{M[AB]}\Sigma^{AB} (Options: Normalize, Confidence=50)
- \frac{\mu_k-\bar{\mu}_k}{\zeta-\mu_k} (Options: Normalize, Confidence=50)
- \frac{B^2}{A^2}-\frac{l^2}{r^2} (Options: Normalize, Confidence=50)
- \int_0^\infty d^D r_\alpha V(r_\alpha) (Options: Normalize, Confidence=50)
- AdS_3 \times S^3 \times S^3 \times S^1 (Options: Normalize, Confidence=50)
Segmentation-Based Search
These examples demonstrate searches that use the Segment
option to automatically decompose the query into structurally meaningful
sub-expressions. When enabled, lower coverage values increase recall by accepting
partial matches, whereas higher values require a closer correspondence to the
original query.
- \Delta=(\begin{array}{c} x-i \lambda 1_2 \\ x+i \lambda 1_2 \end{array}) (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=0, Confidence=50)
- \{ \begin{array}{c} v_t = -u_x \\ u_t = -v^{n-2} v_{x} \end{array} (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=40, Confidence=50)
- \gamma_A = \frac{ \sqrt{2} C(\kappa) \alpha} {24\pi (2\kappa^2-1)^3} (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=30, Confidence=50)
- ( \begin{array}{cc} a1 & b\eta \\ c\eta & d1 \end{array} ), ad-bc=1 (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=20, Confidence=50)
- H_{2m+1} = -\frac{1}{(2m+1) \kappa^m} sTr L^{\frac{2m+1}{2}} (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=20, Confidence=50)
- {( \frac{\dot {a}}{a})} ^2 = \frac{8\pi G}{3} \rho -\frac{k}{a^2} (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=30, Confidence=50)
- Z[s(\sigma)] = {[\underset{n\rightarrow\infty}{\operatorname{lim}} {([s(\sigma)]^{-D \slash 2} e^{ -\frac{m^{2}}{2} \int d^p \sigma s(\sigma)} V )} ^n]}^N (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=30, Confidence=0)
- (\frac {\partial} {\partial \chi} - r + \sigma) (\begin{array}{c} a \\ b \end{array}) = (\begin{array}{c} - (\hat{\phi} - \zeta) h \\ 2 \bar {\partial}_{\hat{A} + \hat{a}} h \end{array}) (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=20, Confidence=0)
Boolean Queries: OR
The OR operator returns pages matching any of the specified
query expressions. When Segment and Segm.Coverage are
used with OR queries, coverage is computed across the combined decomposed segments but
requires those matched segments to occur within the same expression. This can be useful,
but the interpretation may be ambiguous because matching segments may come from different
OR branches.
- n(n+1) OR N(N+1) (Options: Normalize, Confidence=50)
- \sin\theta OR \cos\theta (Options: Normalize, Confidence=50)
- e^{i\vartheta}\sinh\tau OR e^{-i\vartheta}\sinh\tau (Options: Normalize, Confidence=50)
- \frac{n(n-1)}{2} OR \frac{N(N-1)}{2} (Options: Normalize, Segment, Min.Tokens=2, Segm.Coverage=40, Confidence=50)
- \begin{array}{cc} 0 & I \\ - I & 0 \end{array} OR \begin{array}{cc} 0 & I d \\ - I d & 0 \end{array}(Options: Normalize, Segment, Min.Tokens=1, Segm.Coverage=40, Confidence=50)
Boolean Queries: AND
Match pages containing all specified query components. When Segment
and Segm.Coverage are used with AND queries, coverage is computed across the
combined decomposed segments but requires those matched segments to occur within the same
expression. This can be useful, but the interpretation may be ambiguous because matching
segments may come from different AND branches.
- a^2 + b^2 AND c^2 (Options: Normalize, Confidence=50)
- \sin\theta AND \cos\theta (Options: Normalize, Confidence=50)
- e^{i\vartheta}\sinh\tau AND \cosh\tau (Options: Normalize, Confidence=50)
- e^{i\vartheta}\sinh\tau AND e^{-i\vartheta}\sinh\tau (Options: Normalize, Confidence=20)
- \frac{n(n-1)}{2} AND \frac{n(n+1)}{2} (Options: Normalize, Confidence=50)
- \begin{array}{cc} A & B \\ C & D \end{array} AND SO(2,2) (Options: Normalize, Segment, Min.Tokens=1, Segm.Coverage=60, Confidence=50)
Boolean Queries: NOT
The NOT operator returns pages matching the expression on the left,
while reducing or excluding pages that also match the expression on the right. This
exclusion is confidence-based: a weak match on the right may only lower the score and
may not remove the page. The right-hand expression must match strongly enough, relative
to the selected Confidence threshold, to act as an effective veto.
Segm.Coverage is currently disabled for NOT queries because coverage
mode does not yet support exclusion branches.
Grouping and Nested Queries
Parentheses allow multiple Boolean conditions to be combined into a
single query. Grouped expressions are evaluated at the page level,
making it possible to search for pages satisfying complex logical
combinations of mathematical expressions.
Sequence Constraints (SEQ)
Search for expressions whose components appear in a specified
symbolic order. SEQ is a more restrictive form
of the AND operator. All query expressions or
segments must be matched within the same mathematical expression
and must appear in the order specified by the query. The ordering
is determined by the canonical reading order of the mathematical
expression rather than by their visual layout on the page.
Sequence queries must be enclosed within $ACOR and
$CCOR delimiters, which can be inserted using the
corresponding bracket buttons in the query editor.
Query-by-Image
Search using a cropped image of a mathematical expression instead of typing LaTeX.
- Upload an image containing a mathematical expression.
- Select a formula directly from a document page using Recognize from crop.
- Edit the recognized expression before searching.
Indexed Segments
The frequency list shows expression segments that are present in
the index, together with their accumulated score frequency. The list is filtered
to exclude segments with an accumulated score frequency of 1 or less, reducing
noise from one-off fragments while keeping commonly indexed segments inspectable.
This can help inspect which formula fragments are available for direct search,
segmentation, and coverage-based queries.
Open the indexed segments list
Additional Help and Feedback
Questions, bug reports, and feedback about the demonstrator are welcome. For
assistance or to report issues, please contact danitei at prhlt dot
upv dot es.