Netwrix 1Secure 提供跨数据和身份的统一可见性——免费试用14天,享有完全访问权限。开始免费试用

资源中心博客

正则表达式入门:如何开始发现敏感数据

正则表达式入门:如何开始发现敏感数据

Mar 17, 2023

任何数据发现与分类解决方案都高度依赖正则表达式(有时称为 RegExes、REs 或 RegEx patterns),以识别敏感数据。但 RegExes 是什么?又如何利用它们来发现敏感数据呢?让我们一探究竟。

正则表达式(Regular expressions)是一门体量不大但高度专业化的编程语言;本质上就是“比通配符更强的通配符”。使用这种小型语言,你可以指定定义你要匹配的字符串的规则。例如,你可以定义一个 RegEx,用于匹配电子邮件地址、PII、PHI 或信用卡号码。

正则表达式(Regex)组件

一个 RegEx 可以包含字面量(literals)和元字符(metacharacters)。

字面量

除非该字符被保留为元字符(metacharacters),否则任何单个字符本身就已经是一个正则表达式了。例如, www 能匹配 www.Netwrix.com,但 wwz 不能匹配。注意正则表达式是区分大小写的,因此 www 将无法匹配 WWWwWw

元字符

以下这些单个字符不会被解释为字面量,而是具有特殊含义:

  • . ^ $ * + ? { } [ ] | ( )

下表描述了这些元字符各自的功能。

Type

Meta-characters

Description

Examples

The dot

.

The period means any character.

net.rix will match both www.netwrix.com and www.netfrix.com.

Character class

[]

Matches for anything inside the square brackets.
The one exception is the ^ character. Inside a class, at the beginning, the ^ means exception from the search. For example [^n] will match any character except n; this is called a negated character class.
Note that metacharacters (with one exception) are not active inside classes. For example, [net$] will match any of the characters n, e, t or $ ($ is a metacharacter, but inside a character class it matches only $).
The one exception is the ^ character. Inside a class, at the beginning, the ^ means exception from the search. For example [^n] will match any character except n; this is called a negated character class.

You can list characters individually; for instance, net[wrx] will match netw , netr and netx but not netz.
Or you can look for a range of characters by giving two characters and separating them by a hyphen; for example, net[a-z] will match neta, netw and netf but not net1.

Anchors

^

Used to match characters at the beginning of a string

^https will match https://netwrix.com but not www.netwrix.com or http://netwrix.com

$

Used to match characters at the end of a string

com$ will match www.netwrix.com or telecom but not computer.

Iteration / quantifiers

?

Matches the preceding element zero or one time (it will always match if the character was not found). It is great for finding optional characters.

colou?r will match both color and colour.

*

Matches the preceding element zero or more times instead of zero or once. It is great for finding optional series of characters.

ne*t will match nt (zero e characters), net (one e ), neeet (three e characters), and so forth.

+

Matches the preceding element one or more times.
Pay careful attention to the difference between * and +. * matches zero or more times, so whatever’s being repeated may not be present at all; + requires at least one occurrence.

ne+t will match net and neeet but not nt.

|

The choice operator matches either the expression before or the expression after the operator.

net|wrix will match net and wrix.

{}

{x} matches if the element that precedes it is found exactly x times.
{x,y} matches if the preceding element is found at least x times but not more than y times.

n{3} will match nnn , nnnn and nnnd (because they all include n three times in a row), but it will not match nnw.
9{3} will match 999, 1234999124 and text999text, but not 84299238, 9909, or page992.
n{3,5} will match nnn, nnnn and nnnnn.

Blocking and capturing

()

Defines a subexpression that can be recalled later using shorthand: The first subexpression in parentheses can be recalled by \1, the second can be recalled by \2 and so on.
Parentheses are normally used either with | (the choice operator) inside or with quantifiers on the outside.

Gr(a|e)y will match Gray or Grey.
[0-9]([-])[0-9]\1[0-9] will match 3-4-2 and 4-6-1, but not 1-23, 42-1 or 234.

Escape sequence

\

The metacharacter that follows the slash will be used as a literal.
Note that some sequences beginning with \ are not escape sequences. Instead, they represent predefined sets of characters that are often useful, such as the set of digits, the set of letters, or the set of anything that isn’t whitespace. The most popular ones are listed below as “special metacharacters.”

www\.netwrix\.com will match www.netwrix.com but not www,netwrix,com.

Special metacharacters

\s

Matches any whitespace character (a space, a tab, a line break or a form feed).

Netwrix\sAuditor will match Netwrix Auditor, and Netwrix(tab)Auditor, but not Netwrix<5 spaces> Auditor or NetwrixAuditor.

\S

Matches any non-whitespace character.

\Snetwrix will match Xnetwrix and 1netwrix.

\w

Matches any alphanumeric character.

\w\w\w will match net, dfw and Netwrix.

\W

Matches any non-alphanumeric character.

netwrix\W will match netwrix! and netwrix?.

\d

Matches any decimal digit.

Netwrix\d\d will match Netwrix80 and Netwrix90.

\D

Matches any non-digit character.

Netwrix\D will match Netwrix) and Netwrix-.

\a

Matches any single alphabetic character, either capital or lowercase.

net\arix will match netWrix, netfrix and netarix.

\b

Defines a word boundary.

\brix will match rix and rixon but not netwrix.

\B

Defines a non-word boundary

\Brix will match Netwrix and trix but not rixon.

元字符组合

现在我们几乎已经掌握了所有元字符,并准备将它们组合起来使用。

示例:查找车牌号码

假设我们需要找到符合以下格式的车牌号码:aaa-nnnn——前三位必须是字母数字混合,后四位必须是数字。连字符可以被替换为任意字符,也可以完全缺失。

对应的正则表达式(RegEx)将是:

  • b[0-9A-Z]{3}([^ 0-9A-Z]|s)?[0-9]{4}b

让我们来拆解这个正则表达式(RegEx):

  • b 需要一个单词边界,因此匹配到的字符串不能是更长字符串的一部分。
  • [0-9A-Z]{3} 表示前三个字符必须是字母或数字。
  • ([^ 0-9A-Z]|s)? 表示字符串的下一部分必须是定界符——非字母数字字符或空白字符——或者什么都没有。
  • [0-9]{4} 表示字符串的下一部分必须是4位数字。
  • b 指定另一个单词边界。

此 RegEx 将匹配以下许可证号码:NT5-6345, GH3 9452, XS83289

但是,它不会匹配以下许可证号码:ZNT49371, HG3-29347, nt4-9371

示例:查找社会保障号码

另一个很好的例子是美国社会保障号码(SSN),它总是采用以下格式:nnn-nn-nnnn

最简单的正则表达式(RegEx)如下:

  • [0-9]{3}-[0-9]{2}-[0-9]{4}

不过,这会产生误报,因为并非所有符合该格式的数字都是合法的 SSN。此外,它还会漏掉一些真实的 SSN,包括那些没有连字符的写法。为了获得更准确的结果,我们应该构建更复杂的正则表达式。我们知道:

  • 任何数字组都不能全为 0。
  • 第一组不能是 666900-999
  • SSN 可以用空白字符替代连字符来书写,或者完全不使用任何分隔符。
  • 如果第一个块以 7 开头,那么后面必须跟随 06 之间的数字,然后再跟任意第三位数字。

因此,高级 RegEx 将如下所示:

  • b(?!000|666|9d{2})([0-8]d{2}|7([0-6]d))([-]?|s{1})(?!00)dd2(?!0000)d{4}b

和之前一样,b 位于开头和结尾,用于指定单词边界。我们来更深入地看看中间每个数字块。

第一个区块

  • (?!000|666|9d{2}) 是一个否定的前瞻(negative look-ahead),用于指定该数字不得以以下内容开头:0006669,后面跟任意两位数字。
  • ([0-8]d{2} 指定字符串必须以介于 08 之间的数字开头,并且还要再有两位数字 (0-9) 在其后。
  • |7[0-6]d)) 表示它会以 7 开头时,下一位数字必须在 06 之间,后面再跟任意一位数字。
  • ([-]?|s{1}) 指定在三个数字之后,可以是连字符、空白字符,或者什么都不需要,以标记第一个区块的结束。

第二个块

  • (?!00) 也是另一个负向前瞻,用于指定第二个块中不能出现 00
  • dd 表示第二个块中必须有任意两位数字。
  • 2 匹配与第二个捕获组相同的文本,即 ([-]?|s{1}), 因此它指定第二个块可以以连字符、空白字符或完全不带额外字符结束。

第三个块

  • (?!0000) 是另一个负向前瞻(negative look-ahead),用于指定第三个块中不能出现四个 0。
  • d{4} 要求第三个 SSN 块中必须是任意四位数字。

一些常用的正则表达式(RegEx)示例

To find

Use this RegEx

Example of match

Email addresses

^[\w\.=-]+@[\w\.-]+\.[\w]{2,3}$

T.Simpson@netwrix.com

U.S. Social Security numbers

\b(?!000|666|9\d{2})([0-8]\d{2}|7([0-6]\d))([-]?|\s{1})(?!00)\d\d\2(?!0000)\d{4}\b

513-84-7329

IPV4 addresses

^\d{1,3}[.]\d{1,3}[.]\d{1,3}[.]\d{1,3}$

192.168.1.1

Dates in MM/DD/YYYY format

^([1][12]|[0]?[1-9])[\/-]([3][01]|[12]\d|[0]?[1-9])[\/-](\d{4}|\d{2})$

05/05/2018

MasterCard numbers

^(?:5[1-5][0-9]{2}|222[1-9]|22[3-9][0-9]|2[3-6][0-9]{2}|27[01][0-9]|2720)[0-9]{12}$

5258704108753590

Visa card numbers

\b([4]\d{3}[\s]\d{4}[\s]\d{4}[\s]\d{4}|[4]\d{3}[-]\d{4}[-]\d{4}[-
]\d{4}|[4]\d{3}[.]\d{4}[.]\d{4}[.]\d{4}|[4]\d{3}\d{4}\d{4}\d{4})\b

4563-7568-5698-4587

American Express card numbers

^3[47][0-9]{13}$

34583547858682157

U.S. ZIP codes

^((\d{5}-\d{4})|(\d{5})|([A-Z]\d[A-Z]\s\d[A-Z]\d))$

97589

File paths

\\[^\\]+$

\\fs1\shared

URLs

(?i)\b((?:[a-z][\w-]+:(?:\/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}\/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+
|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?«»“”‘’]))

www.netwrix.com

有用的正则表达式(Regex)网页资源

分享到

了解更多

关于作者

Asset Not Found

Jeff Melnick

系统工程总监

Jeff 是 Netwrix 的前 Global Solutions Engineering 总监。他是一位长期的 Netwrix 博主、演讲者和讲解员。在 Netwrix 博客中,Jeff 分享各种生活技巧,以及可以显著提升您系统管理体验的提示与技巧。