xenia/kakoune
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,192 Commits 1 Branch 0 Tags 17 MiB
6fdb9db5d9
Commit Graph

113 Commits

Author SHA1 Message Date
Maxime Coste
fb65fa60f8 Regex: take the full subject range as a parameter 
To allow more general look arounds out of the actual search range,
pass a second range (the actual subject). This allows us to remove
various flags such as PrevAvailable or NotBeginOfSubject, which are
now easy to check from the subject range.

Fixes #1902
 2018-03-05 05:48:10 +11:00
Maxime Coste
933ac4d3d5 Regex: Improve comments and constify some variables 
Reword various comments to make some tricky parts of the regex
engine easier to understand.
 2018-02-24 17:40:08 +11:00
Maxime Coste
3584e00d19 Regex: Use a template argument instead of a regular one for "forward" 
forward (which controls if we are compling for forward or backward
matching) is always statically known, and compilation will first
compile forward, then backward (if needed), so by having separate
compiled function we get rid of runtime branches.
 2018-02-09 22:45:53 +11:00
Maxime Coste
aa9f7753e8 Regex: minor code cleanup 2018-02-09 22:19:56 +11:00
Maxime Coste
413f880e9e Regex: Support forward and backward matching code in the same CompiledRegex 
No need to have two separate regexes to handle forward and backward
matching, just passing RegexCompileFlags::Backward will add support
for backward matching to the regex. For backward only regex, pass
RegexCompileFlags::NoForward as well to disable generation of
forward matching code.
 2017-12-01 19:57:02 +08:00
Maxime Coste
7bfb695c45 Regex: Do not allow private use codepoints literals 
We use them to encode non-literals in lookarounds, so they can
trigger bugs.

Fixes #1737
 2017-12-01 16:37:18 +08:00
Maxime Coste
65b057f261 Regex: rename StartChars to StartDesc 
It only contains chars for now, but its still more generally
describing where matches can start.
 2017-12-01 14:46:18 +08:00
Maxime Coste
b91f43b031 Regex: optimize parsing a bit 2017-11-30 14:32:29 +08:00
Maxime Coste
c1f0efa3f4 Regex: smarter handling of start chars computation for character class 2017-11-30 14:19:41 +08:00
Maxime Coste
ae0911b533 Regex: Various small code tweaks 2017-11-28 01:03:54 +08:00
Maxime Coste
4598832ed5 Regex: optimize compilation by reserving data 2017-11-28 00:59:57 +08:00
Maxime Coste
a52da6fe34 Regex: Tweak is_ctype implementation style 2017-11-28 00:13:42 +08:00
Maxime Coste
8b40f57145 Regex: Replace generic 'Matchers' with specialized functionality 
Introduce CharacterClass and CharacterType Regex Op, and optimize
their evaluation.
 2017-11-25 18:14:15 +08:00
Maxime Coste
0d44cf9591 Regex: do not decode utf8 in accept calls as they always run on ascii 2017-11-25 18:13:27 +08:00
Maxime Coste
ffb639bf96 Regex: add unit test for #1693 2017-11-13 01:12:05 +08:00
fsub
0dd8a9ba93 Fix #1693: typo in RegexParser::character_class() 2017-11-12 17:35:03 +01:00
Maxime Coste
f07375fb27 Regex: remove dead code 2017-11-01 14:05:15 +08:00
Maxime Coste
2c2073b417 Regex: Tweak struct layouts of ParsedRegex data 2017-11-01 14:05:15 +08:00
Maxime Coste
bbd7e604dc Regex: Remove "Ast" from names in the ParsedRegex 
It does not add much value, and makes names longer.
 2017-11-01 14:05:15 +08:00
Maxime Coste
18a02ccacd Regex: Optimize parsing and compilation 
AstNodes are now POD, stored in a single vector, accessed through
their index. The children list is implicit, with nodes storing only
the node index at which their child graph ends.

That makes reverse iteration slower, but that is only used for reverse
matching regex, which are uncommon. In the general case compilation
is now faster.
 2017-11-01 14:05:15 +08:00
Maxime Coste
aea2de885d Regex: minor cleanup of the regex parsing code 2017-11-01 14:05:15 +08:00
Maxime Coste
6e0275e550 Regex: small code cleanup in the Save compilation code 2017-11-01 14:05:15 +08:00
Maxime Coste
9e15207d2a Regex: put the other char boolean inside the general start char map 2017-11-01 14:05:15 +08:00
Maxime Coste
60e32d73ff Regex: Fix handling of all unicode codepoint as start chars 2017-11-01 14:05:15 +08:00
Maxime Coste
df2bf9601c Regex: fix wrong fallthough in dump_regex 2017-11-01 14:05:15 +08:00
Maxime Coste
d9b4076e3c Regex: Go back to instruction based search of next start 
The previous method, which was a bit faster in the general use case,
can hit some cases where we get quadratic behaviour and very slow
matching.

By using an instruction, we can guarantee our complexity of O(N*M)
as we will never have more than N threads (N being the instruction
count) and we run the threads once per codepoint in the subject
string.

That slows down the general case slightly, but ensure we dont have
pathological cases.

This new version is much faster than the previous instruction based
search because it does not use a plain `.*` searcher, but a specific,
smarter instruction specialized for finding the next start if we are
in the correct conditions.
 2017-11-01 14:05:15 +08:00
Maxime Coste
3f627058b0 Regex: add support for \0, \cX, \xXX and \uXXXX escapes 2017-11-01 14:05:15 +08:00
Maxime Coste
c423b47109 Regex: compute if codepoints outside of the start chars map can start 2017-11-01 14:05:15 +08:00
Maxime Coste
2c6c0be0c1 Regex: abort compilation as soon as we hit the instruction count limit 2017-11-01 14:05:15 +08:00
Maxime Coste
d44e160aa7 Regex: add a unit test for why lookaheads dont count for start chars anymore 2017-11-01 14:05:15 +08:00
Maxime Coste
87eec79d07 Regex: comment the mutables in CompiledRegex::Instruction and fix their init 2017-11-01 14:05:14 +08:00
Maxime Coste
8b2297f5ca Regex: Introduce a Regex memory domain to track usage separately 2017-11-01 14:05:14 +08:00
Maxime Coste
9ec175f2f8 Regex: use binary search to for character class ranges check 2017-11-01 14:05:14 +08:00
Maxime Coste
6e65589a34 Regex: compute start chars from matchers, do not compute it from lookarounds 
Computing potential start characters from lookarounds is more complex
than expected, and not worth the complexity.
 2017-11-01 14:05:14 +08:00
Maxime Coste
df16fea82d Regex: rename "flags" with the more common "modifiers" 2017-11-01 14:05:14 +08:00
Maxime Coste
52d443f764 Regex: Correctly handle ignore case mode for start chars computation 2017-11-01 14:05:14 +08:00
Maxime Coste
b8495f0953 Regex: Rework parsing, treat lookarounds as assertions, and flags separately 2017-11-01 14:05:14 +08:00
Maxime Coste
b0233262b8 Regex: Limit programs to std::numeric_limits<uint16_t>::max() instructions 2017-11-01 14:05:14 +08:00
Maxime Coste
8c8dcb3a84 Regex: Fix reverse searching behaviour, again 2017-11-01 14:05:14 +08:00
Maxime Coste
9753bcd0ad Regex: limit explicit quantifiers value (too 1000 for now) 
Fixes #1628
 2017-11-01 14:05:14 +08:00
Maxime Coste
2b97e4e124 Regex: Fix handling of ^ and $ in backward matching mode 2017-11-01 14:05:14 +08:00
Maxime Coste
5bf4be645a Regex: Fix support for ignore case in lookarounds 2017-11-01 14:05:14 +08:00
Maxime Coste
23b3a221eb Regex: support more than two children in alternations 
Avoid deep nested alternations, parse them flattened.
 2017-11-01 14:05:14 +08:00
Maxime Coste
fb5243f710 Regex: print instruction index in dump_regex 2017-11-01 14:05:14 +08:00
Maxime Coste
74ed102cab Regex: Tweak definition of character class and control escape tables 2017-11-01 14:05:14 +08:00
Maxime Coste
df73b71dfc Regex: fix lookarounds handling when computing starting chars 2017-11-01 14:05:14 +08:00
Maxime Coste
785cd34b4b Regex: Make boost checking disableable at compile time 2017-11-01 14:05:14 +08:00
Maxime Coste
065bbc8f59 Regex: switch to custom impl, use boost for checking 2017-11-01 14:05:14 +08:00
Maxime Coste
9305fa1369 Regex: Fix lookaround use in moon.kak 
(?=[A-Z]\w*) is strictly the same as (?=[A-Z]) as \w* will always
at least match an empty string.
 2017-11-01 14:05:14 +08:00
Maxime Coste
cca730193c Regex: Support any char and character classes in lookarounds 
Lookarounds still need to be fixed size, but accept character classes
as well as plain literals.
 2017-11-01 14:05:14 +08:00