Skip to content
InvestShield

InvestShield / The technology

Detection technology

Seven stages connect a public signal to a documented detection. For each stage: the input, the operation and the result available to the analyst.

See the data

The seven stages of detection

Each step records what it received and what it produced, with the date and time. The case file of a detection can be rebuilt step by step, and every measure on this site comes from a repeatable query over that data.

  1. Observing the logs

    We continuously read public certificate logs. Entry coordinates are available when provenance is attested.

    Input
    Certificate Transparency logs
    Output
    Observed domains and log references, when present
    Detail and measuresObserving the logs

    Certificate Transparency logs are public, append-only lists in which certificate authorities enter every TLS certificate they issue. The system continuously reads the union of the logs recognised by Chrome and by Apple. Every entry carries the log identifier, the leaf index and the leaf hash: the coordinates with which anyone can find it again in the public log.

    Coverage is verified on distinct certificates: by policy every certificate is entered in at least two logs run by different operators, so the logs we read cover for one another.

    Certificate Transparency logs are public, append-only lists in which certificate authorities enter every TLS certificate they issue. The system continuously reads the union of the logs recognised by Chrome and by Apple. Every entry carries the log identifier, the leaf index and the leaf hash: the coordinates with which anyone can find it again in the public log.

    Coverage is verified on distinct certificates: by policy every certificate is entered in at least two logs run by different operators, so the logs we read cover for one another.

  2. First filter

    We select financially relevant names, distinguishing registrable domains from hosting subdomains.

    Input
    Observed domain names
    Output
    Domains selected for reading
    Detail and measuresFirst filter

    On the domain name: an index of financial keywords in several languages and an index of the names cited in authorities’ warnings, read through the Public Suffix List so that a subdomain of a hosting service is not mistaken for a domain. The filter keeps roughly one domain in a thousand of those seen in a day.

    Measured on the production registry: between 0.10% and 0.13% of the distinct domains seen in a day. The registry also keeps what was retained without becoming a case: “never seen” and “seen and discarded” are two different pieces of information.

    On the domain name: an index of financial keywords in several languages and an index of the names cited in authorities’ warnings, read through the Public Suffix List so that a subdomain of a hosting service is not mistaken for a domain. The filter keeps roughly one domain in a thousand of those seen in a day.

    Measured on the production registry: between 0.10% and 0.13% of the distinct domains seen in a day. The registry also keeps what was retained without becoming a case: “never seen” and “seen and discarded” are two different pieces of information.

  3. Reading the page

    We observe the page, its redirects and the languages it actually serves, checked against the content.

    Input
    Selected domains
    Output
    Content, final address, HTTP response and observed languages
    Detail and measuresReading the page

    The page is requested following redirects; the response code, final address, response time and the languages actually served are recorded. A language counts as served if the page is in that language or a version in that language was observed, not merely declared.

    Median reading time in the window: 0.7 s.

    The page is requested following redirects; the response code, final address, response time and the languages actually served are recorded. A language counts as served if the page is in that language or a version in that language was observed, not merely declared.

    Median reading time in the window: 0.7 s.

  4. Classification

    We extract services, declared entities and indicators from the text. Uncertain cases move to the second-tier model.

    Input
    Page text
    Output
    Indicators, model used and extracted fields
    Detail and measuresClassification

    A two-tier language model reads the text: a fast first tier on every page, a more capable second tier only when the first tier signals uncertainty. It extracts the type of service, the authorisations claimed, the legal entity, the contact channels and the indicators: guaranteed returns, authorisation not stated, deposit pressure, artificial urgency, contact only through private chat. Every verdict records the model that produced it. Claimed authorisations are recorded as text, with the page sentence that states them: matching them against official registers is left to the supervising authority, except for a list of Italian intermediaries’ sites, including those notified by investment firms (SIM) on the CONSOB register, which keeps a registered intermediary’s site from being mistaken for a domain cited in a warning.

    In the window: 28,127 verdicts, 8,770 escalated to the second tier (31%); 3.2 s median classification time. The model works with an explicit framing: “analyst for the Italian regulator”.

    A two-tier language model reads the text: a fast first tier on every page, a more capable second tier only when the first tier signals uncertainty. It extracts the type of service, the authorisations claimed, the legal entity, the contact channels and the indicators: guaranteed returns, authorisation not stated, deposit pressure, artificial urgency, contact only through private chat. Every verdict records the model that produced it. Claimed authorisations are recorded as text, with the page sentence that states them: matching them against official registers is left to the supervising authority, except for a list of Italian intermediaries’ sites, including those notified by investment firms (SIM) on the CONSOB register, which keeps a registered intermediary’s site from being mistaken for a domain cited in a warning.

    In the window: 28,127 verdicts, 8,770 escalated to the second tier (31%); 3.2 s median classification time. The model works with an explicit framing: “analyst for the Italian regulator”.

  5. Effective severity

    We combine indicators with popularity, redirects and warnings. Severity is a technical measure and also depends on warning matches.

    Input
    Indicators and domain checks
    Output
    Effective severity for the analysis queue
    Detail and measuresEffective severity

    The intrinsic severity, computed from the indicators, is corrected with what the content alone cannot say: the domain’s popularity in the public Chrome UX Report sample, redirects to known sites, matches with the names cited in warnings. The result is the effective severity, the only one used on this site and in the work queue.

    The match with warnings also enters the computation, and the Data page accounts for this when it compares severity with warnings.

    The intrinsic severity, computed from the indicators, is corrected with what the content alone cannot say: the domain’s popularity in the public Chrome UX Report sample, redirects to known sites, matches with the names cited in warnings. The result is the effective severity, the only one used on this site and in the work queue.

    The match with warnings also enters the computation, and the Data page accounts for this when it compares severity with warnings.

  6. Preservation

    We capture pages and resources in WARC format for domains at elevated severity, with fingerprint and capture time.

    Input
    Page and resources accessible to the crawler
    Output
    Dated copy with SHA-1 fingerprints, when captured
    Detail and measuresPreservation

    For every domain at elevated severity, a crawler that identifies itself as such captures the page and its resources in WARC format, the web-archiving standard, with a content fingerprint for every record and the capture date and time. Copies are viewed in a separate viewer that runs the preserved HTML in isolation from the console.

    Every copy, and every missing copy, stays recorded in the case file; the measured coverage is in the evidence document.

    For every domain at elevated severity, a crawler that identifies itself as such captures the page and its resources in WARC format, the web-archiving standard, with a content fingerprint for every record and the capture date and time. Copies are viewed in a separate viewer that runs the preserved HTML in isolation from the console.

    Every copy, and every missing copy, stays recorded in the case file; the measured coverage is in the evidence document.

  7. Reachability check

    We check whether the server still responds: a 403 or 500 also indicates an active service.

    Input
    Domains subject to periodic checks
    Output
    Reachability status and check date
    Detail and measuresReachability check

    A periodic check asks the server whether it still responds. A 403 or 500 code counts as active: it is usually a protection service in front of the site. A domain never checked stays distinct from a reachable one, on screen as well.

    The shares and outcomes of the window’s checks are on the Data page.

    A periodic check asks the server whether it still responds. A 403 or 500 code counts as active: it is usually a protection service in front of the site. A domain never checked stays distinct from a reachable one, on screen as well.

    The shares and outcomes of the window’s checks are on the Data page.