this post was submitted on 09 Feb 2024
91 points (88.9% liked)
Programming
17344 readers
150 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Any time you're turning a string of input into something else, what you are doing is parsing.
Even if the word "parser" never appears in your code, the act of interpreting a string as structured data is parsing, and the code that does parsing is a parser.
Programmers write parsers quite a lot, and many of the parsers they write are ad-hoc, ill-specified, bug-ridden, and can't tell you why your input didn't parse right.
Writing a parser without realizing you're writing a parser, usually leads to writing a bad parser. Bad parsers do things like accepting malformed input that causes security holes. When bad parsers do reject malformed input, they rarely emit useful error messages about why it's malformed. Bad parsers are often written using regex and duct tape.
Try not to write bad parsers. If you need to parse something, consider writing a grammar and using a parser library. (If you're very ambitious, try a parser combinator library.) But at least try to recall something about parsers you learned once way back in a CS class, before throwing regex at the problem and calling it a day.
(And now the word "parser" no longer makes sense, because of semantic satiation.)
By the way, please don't write regex to try to validate email addresses. Seriously. There are libraries for that; some of them are even good. When people write their own regex to match email addresses, they do things like forget that the hyphen is a valid character in domain names.
Amen.
Spoiler alert: Few of them are good, and those few are so simple that you might as well not use a library.
The only way to correctly validate an email address is to send a message to it, and verify that it arrived.
You can use a regex to do basic validation. That regex is
.+@.+. Anything beyond that is a waste of time.That rejects valid emails
Which ones? In RFC 5322 every address contains an addr-spec at some point, which in turn must include an @. RFC 6854 does not seem to change this. Or did I misread something?
EDIT: I'M DUMB I THOUGHT YOU MEANT A LITERAL "." , that regex is indeed correct.
Original:
This is a valid address:
user.name@[IPv6:2001:db8:1ff::a0b:dbd0]Relevant spec: https://www.rfc-editor.org/rfc/rfc5321#section-4.1.3
And you all doubted me... :P
Edit: Wait, I also don't think local-part even needs a "." in it? user@domain.com should be valid as well.
And it's matched by
.+@.+as it contains an @.Remember, we're taking about regular expressions here so
.+means "a sequence of one or more arbitrary characters". It does not imply that an actual dot is present.(And I overlooked the edit. Oops.)
There are also cases where you want to have a disallow list of known bad email providers. That’s also part of the parsing and validating.
It's a valid need, but a domain blacklist isn't part of email parsing and if you conflate the two inside your program then you're mixing concerns.
Why is the domain blacklist even in your program? It should be a user configurable file or a list of domains in the database.
You are right in that it isn't (or shouldn't be) part of the parsing, but the program has to check the blacklist even if it's in a database.
We were discussing validation, not parsing. There’s no parsing in an email. You might give it a type once it passes validation, but an email is just a string with an
@in it (and likely some . because you want at least 1 TLD but even that I’m not sure).fuck any website that requires an account to just READ it's stupid content and at the same time blocks guerrillamail/10minutemail (looking at you, Glassdoor,I don't want to get fucking spam just so that I can check approximate salary in a company)
Sounds like your gripe is with people requiring accounts for reading public content, and not with preventing usage of automated email creation and trying to limit bots on your website.
Imagine giving someone your phone number, and having them say you have to get a different one because they don't like some of the digits in it.
I have seen this nonsense more times than I care to remember. Please don't build systems this way.
If you're trying to do bot detection or the like, use a different approach. Blacklisting email addresses based on domain or any other pattern does a poor job of it and creates an awful user experience.
(And if it prevents people from using spam-fighting tools like forwarding services, then it's directly user-hostile, and makes the world a worse place.)
If you're accepting email addresses as user input (e.g. from a web form), it might be nice to check that what's to the right of the rightmost
@sign is a domain name with an MX or A record. That way, if a user enters a typo'd address, you have some chance of telling them that instead of handing an email touser#example.netoruser@gmailc.omto your MTA.But the validity of the local-part (left of the rightmost
@) is up to the receiving server.Checking MX in your application means you needlessly fail on transient outages, like when a DNS server is rebooting or a net link hiccups. When it happens, the error flag your app puts on the user's email address is likely to confuse or frustrate them, will definitely waste their time, and may drive them away and/or generate support calls.
Also, MX records are not required. Edit to clarify: So checking MX in your application means you fail 100% of the time on some perfectly valid email domains. Good luck to the users and support staff who have to troubleshoot that, because there's nothing wrong with the email address or domain; the problem is your application doing something it should not.
Better to just hand the verification message off to your mail server, which knows how to handle these things. You can flag the address if your outgoing mail server refuses to accept it.
If DNS is transiently down, the most common mail domains are still in local resolver cache. And if you're parsing live user requests, that means the IP network itself is not in transient failure at the moment. So it takes a pretty narrow kind of failure to trigger a problem... And the outcome is the app tells the user to recheck their email address, they do, and they retry and it works.
If DNS is having a worse problem, it's probably down for your mail server too, which means an email would at least sit in the outbound mail spool for a bit until DNS comes back. Meanwhile the user is wondering where their confirmation email is, because people expect email delivery in seconds these days.
So yeah ... yay, tradeoffs!
(Confirmation emails are still important for closed-loop opt-in, to make sure the user isn't signing someone else up for your marketing department's spam, though.)
Speaking of things you can't parse with regex...
The best validation for a valid email address is always sending a verification mail. I've rejected countless MRs that contained ad-hoc regex email validation copied from the internet. I'd allow a check for it to contain an
@.What if it only contains @?
If they are also sending a validation email, it would fail, so no issue.
Honestly I feel like PHP regex may be able to parse html but not entirely sure
Speak for yourself. I've done it exactly once. It didn't work, and never shipped. Learned my lesson and always use a parser that someone else wrote. Usually a big team of at least thousands of people (how many people have worked on JSON? Millions? What about UTF8? Those are the main two I use).
So you have never iterated over command line arguments and tried to identify options? Or taken a string input field?