could someone tell me how the Query Helper Processor works internally or more specifically whether it is able to leverage Spark for parallel computations?
My specific use case is: I have a (potentially huge, millions of rows) table with key-value pairs. I want to apply queries which perform an RLIKE operation on the values, that means they will evaluate whether the values fit a regular expression. There can be multiple queries depending on the amount of regular expressions that are fed into the workflow via another input and can be quite a few as well (expected in a two-digit range).
Now my question more specifically: Can the Query Helper perform those queries in parallel so that all (or at least some) RLIKE queries are performed in parallel?