`strings` — text

strings is text manipulation. Twenty-nine functions, and the organising convention is worth learning once: a leading I means case-insensitive. Contains and IContains, PhraseCount and IPhraseCount, and so on.

On extracted documents, prefer the I forms unless a case difference is itself meaningful. OCR capitalisation is not evidence of anything.

Case and trimming

strings.ToUpper(s) → string · strings.ToLower(s) → string · strings.ToTitle(s) → string

Parameter	Type	Meaning
s	string	The text to convert

ToTitle capitalises the first letter of each word — useful for normalising a name read from an all-caps form before it lands in output.

rule NormaliseBorrowerName "store the name in title case" {
    when
        bbox.Doc().Right("Borrower Name").Found()
    then
        out.Set("borrowerName",
            strings.ToTitle(strings.ToLower(bbox.Doc().Right("Borrower Name").Str())));
}

Lowering first is what makes this work on "JOHN SMITH" — ToTitle alone leaves it unchanged.

strings.TrimSpace(s) → string

Removes leading and trailing whitespace.

Parameter	Type	Meaning
s	string	The text to trim

strings.Len(s) → int64

The length of the text.

Parameter	Type	Meaning
s	string	The text to measure

Substring tests

strings.Contains(s, substr) → bool · strings.IContains(s, substr) → bool

Reports whether s contains substr. IContains ignores case.

Parameter	Type	Meaning
s	string	The text to search
substr	string	What to look for

rule TrustDeedPresent "the document mentions a deed of trust" {
    when
        strings.IContains(bbox.Doc().AllText().Str(), "deed of trust")
    then
        out.Set("instrumentType", "deed_of_trust");
}

strings.HasPrefix(s, prefix) → bool · strings.HasSuffix(s, suffix) → bool

Parameter	Type	Meaning
s	string	The text to test
prefix / suffix	string	What to test for

rule ServiceAccount "identifiers starting with svc- are service accounts" {
    when
        strings.HasPrefix(in.Str("actor.id", ""), "svc-")
    then
        out.Set("actorKind", "service");
}

Splitting, joining, replacing

strings.Split(s, sep) → list of string

Parameter	Type	Meaning
s	string	The text to split
sep	string	The separator

strings.Join(elems, sep) → string

Parameter	Type	Meaning
elems	list of string	The pieces
sep	string	The separator to place between them

strings.Replace(s, old, new) → string

Replaces every occurrence of old with new.

Parameter	Type	Meaning
s	string	The text
old	string	What to replace
new	string	What to replace it with

strings.ReplaceChars(s, replacement, charset) → string

Replaces characters rather than a substring: every character present in charset becomes replacement, and everything else is kept.

Parameter	Type	Meaning
s	string	The text
replacement	string	What each matched character becomes
charset	string	The set of characters to replace

The cleaning function for identifiers read off a document, where the separators vary run to run.

rule NormaliseSSN "strip separators from the tax identifier" {
    when
        bbox.Doc().Right("SSN").Found()
    then
        out.Set("ssn",
            strings.ReplaceChars(bbox.Doc().Right("SSN").Str(), "", "- ."));
}

Regular expressions

Four forms, and choosing the right one saves a lot of post-processing.

strings.MatchRegex(pattern, s) → bool

Reports whether pattern matches anywhere in s.

Parameter	Type	Meaning
pattern	string	The regular expression
s	string	The text to test

rule LoanNumberWellFormed "loan numbers are three letters then eight digits" {
    when
        out.Has("loanNumber") &&
        !strings.MatchRegex("^[A-Z]{3}[0-9]{8}$", out.Str("loanNumber", ""))
    then
        findings.Error("LoanNumberWellFormed", "format", "Loan number does not match the expected shape");
}

strings.FindRegex(pattern, s) → string

Returns the first match, or the empty string when there is none.

Parameter	Type	Meaning
pattern	string	The regular expression
s	string	The text to search

FindRegex("\\d+", "order 12345 confirmed") returns "12345".

strings.FindRegexGroups(pattern, s) → map

Returns the named capture groups of the first match, as a map. Unnamed groups are ignored, and no match returns an empty map.

Parameter	Type	Meaning
pattern	string	The regular expression, with (?P<name>...) groups
s	string	The text to search

The form to reach for when one read yields several fields.

rule SplitPhoneNumber "break the phone number into its parts" {
    when
        bbox.Doc().Right("Phone").Found()
    then
        out.Set("phoneParts",
            strings.FindRegexGroups("(?P<area>[0-9]{3})[-. ](?P<exchange>[0-9]{3})[-. ](?P<line>[0-9]{4})",
                bbox.Doc().Right("Phone").Str()));
}

strings.ExtractRegex(pattern, s, sep) → string

Finds all captures — named or unnamed — across all matches, and joins them with sep.

Parameter	Type	Meaning
pattern	string	The regular expression
s	string	The text to search
sep	string	The separator to join captures with

ExtractRegex("(\\d+)", "order 123 item 456", " ") returns "123 456".

strings.ReplaceRegex(pattern, s, replacement) → string

Replaces all matches. $1, $2 refer to positional groups and ${name} to named ones.

Parameter	Type	Meaning
pattern	string	The regular expression
s	string	The text
replacement	string	The replacement, which may reference groups

rule ReformatDate "turn MM/DD/YYYY into ISO" {
    when
        strings.MatchRegex("^[0-9]{2}/[0-9]{2}/[0-9]{4}$", in.Str("raw.date", ""))
    then
        out.Set("date",
            strings.ReplaceRegex("^(?P<m>[0-9]{2})/(?P<d>[0-9]{2})/(?P<y>[0-9]{4})$",
                in.Str("raw.date", ""), "${y}-${m}-${d}"));
}

Counting phrases

A phrase here may contain several words, which is what separates these from the substring tests: they are built for looking for real language in a body of extracted text.

strings.PhraseCount(text, phrase) → int64 · strings.IPhraseCount(text, phrase) → int64

How many times one phrase occurs.

Parameter	Type	Meaning
text	string	The text to search
phrase	string	The phrase to count

strings.CountPhrases(text, phrases...) → int64 · strings.ICountPhrases(text, phrases...) → int64

The total occurrences of several phrases — a sum across all of them, not a count of how many were present.

Parameter	Type	Meaning
text	string	The text to search
phrases	string, variadic	The phrases to count

rule HeavilyRedacted "many redaction markers suggest a redacted copy" {
    when
        strings.ICountPhrases(bbox.Doc().AllText().Str(), "REDACTED", "[REMOVED]", "XXXXX") > 10
    then
        findings.Warn("HeavilyRedacted", "provenance", "Document appears heavily redacted");
}

strings.HasAllPhrases(text, phrases...) → bool · strings.IHasAllPhrases(text, phrases...) → bool

Reports whether every phrase is present.

Parameter	Type	Meaning
text	string	The text to search
phrases	string, variadic	The phrases that must all appear

rule ClosingDisclosureSections "the form carries all its required sections" {
    when
        strings.IHasAllPhrases(bbox.Doc().AllText().Str(),
            "Loan Terms", "Projected Payments", "Costs at Closing")
    then
        out.Set("formComplete", true);
}

For a presence test on the document itself, bbox.Doc().HasAll(...) is usually better — it matches fuzzily, absorbing the OCR variation these exact-match functions will not.

Locating a phrase

strings.PhraseLineNum(text, phrase) → int64 · strings.IPhraseLineNum(text, phrase) → int64

The 1-based line number where the phrase first appears, or 0 when it is not found.

Parameter	Type	Meaning
text	string	The text to search
phrase	string	The phrase to locate

Zero means “not found” — it is not a line number, since lines start at one.

strings.PhraseLineNums(text, phrase) → list of int64 · strings.IPhraseLineNums(text, phrase) → list of int64

Every 1-based line number where the phrase appears.

Parameter	Type	Meaning
text	string	The text to search
phrase	string	The phrase to locate

rule SignatureBlockPosition "the signature block should be near the end" {
    when
        strings.IPhraseLineNum(bbox.Doc().AllText().Str(), "IN WITNESS WHEREOF") > 0 &&
        strings.IPhraseLineNum(bbox.Doc().AllText().Str(), "IN WITNESS WHEREOF") <
            math.Round(to.Num(bbox.Doc().LineCount()) * 0.5, 0)
    then
        findings.Warn("SignatureBlockPosition", "structure",
            "Signature block appears in the first half of the document");
}

Formatting

strings.Sprintf(format, args...) → string

Formats a string.

Parameter	Type	Meaning
format	string	The format, with %s, %d, %f and the usual verbs
args	any, variadic	The values

Mostly used to build a readable finding message.

rule AmountOverLimit "message names the actual figure" {
    when
        to.Num(out.Get("loanAmount")) > 500000.0
    then
        findings.Warn("AmountOverLimit", "underwriting",
            strings.Sprintf("Loan amount %s exceeds the 500,000 limit",
                to.Grouped(out.Get("loanAmount"), 2)));
}

Prefer to.Grouped / to.Fixed for the numbers inside a message rather than %f, which prints float noise.

See also

• to — conversion — turning text into numbers and dates
• vocab — named lists — keeping phrase lists out of rules
• The Document API — fuzzy matching, which these functions do not do