Add more benchmarks and benchmarker functionality #505
Conversation
| if let otherVal = other.results[item.key] { | ||
| let diff = item.value - otherVal | ||
| // note: is this enough time difference? | ||
| if diff.abs() > Time.millisecond { |
There was a problem hiding this comment.
We don't normalize our execution times, and some of them run in under a millisecond total. Should we be doing something relative?
There was a problem hiding this comment.
Or we may want to remove this just for now and let the user decide. We're not doing long-term perf tracking with this (at least not yet), but it's very useful for development-cycle perf testing.
| benchmark.addGraphemeBreak() | ||
| benchmark.addHangulSyllable() | ||
| benchmark.addHTML() | ||
| benchmark.addEmail() |
There was a problem hiding this comment.
Is there any way that the benchmarks can add themselves instead of having a second listing?
| } | ||
|
|
||
| extension NSBenchmark { | ||
| public func debug() { |
There was a problem hiding this comment.
What does the workflow look like for using this?
There was a problem hiding this comment.
Since the benchmarker normally blackholes the output of the regex it made it easy for regexes that had typos or mistakes in them to go unnoticed (like the css regex)
The debug mode just prints out the match that it found or the number of matches and that lets us make sure the the regexes are at least running somewhat correctly, for example by checking that the number of matches is the same across swift and ns regex
I was using it when adding the html and email regexes by just running something like ./RegexBenchmark --debug emailLookaheadFirst
| let emailRFC = #"[A-z0-9!#$%&'*+\/=?^_‘{|}~-]+(?:\.[A-z0-9!#$%&'*+\/=?^_‘{|}~-]+)*@(?:[A-z0-9](?:[A-z0-9-]*[A-z0-9])?\.)+[A-z0-9](?:[A-z0-9-]*[A-z0-9])?"# | ||
|
|
||
| // More complex, does length and consecutive hyphen validation via lookaheads | ||
| let emailWithLookaheads = #"(?=[A-z0-9][A-z0-9@._%+-]{5,253})[A-z0-9._%+-]{1,64}@(?:(?=[A-z0-9-]{1,63}\.)[A-z0-9]+(?:-[A-z0-9]+)*\.){1,8}[A-z]{2,63}"# |
| @@ -0,0 +1,22 @@ | |||
| import string | |||
There was a problem hiding this comment.
Add a comment with the command-line invocation to populate the output folder file.
|
@swift-ci please test |
|
Merging to unblock some other perf work. |
Adds debug mode (show matches/ # of matches), the ability to save/compare results, and benchmarks for email and html regexes
Is based on top of #501