[Zeek] Certificate questions
Michał Purzyński
michalpurzynski1 at gmail.com
Mon Nov 18 16:36:22 PST 2019
Answers inline. Keep asking, this is tricky to get right.
On Mon, Nov 18, 2019 at 10:44 AM <jayf at wheeling-nisshin.com> wrote:
> Thanks for the answer. I need a bit of clarification though.
>
> Your instructions said, openssl s_client -host slscr.update.microsoft.com
> -port 443 -showcerts < /dev/null | sed -n '/BEGIN/,/END/p' | openssl x509
> -outform DER > o.der
> (do that for each CA - ignore the "verify error")'
>
> Wouldn't fetching each certificate overwrite "o.der". Should that be a
> ">>"?
>
It will and that's why you want to redirect output for each server you're
fetching certificate chain from, to a new file.
gen_certs.py can then consume entire directory full of "der" files and
combined that into a hex representation for a Zeek script.
> Or do need to modify "o.der" for each certificate I fetch, then do a
> "python ../gen_certs.py . cacert.zeek"? Would that grab all the .der
> files? Do I need to make a seperate, unique "cacert.zeek" to put in my
> "@load" statement.
>
So I just tested it a bit more and I need to modify my original instructions
openssl s_client -host init.push.apple.com -port 443 -showcerts < /dev/null
For each server inspect the output carefully - you want to grab the "CA"
and that's usually the last one. Ignore the sed, etc here, it won't,
unfortunately, work as I wanted it to.
For example, Apple returns
CONNECTED(00000005)
---
Certificate chain
0 s:/CN=init.push.apple.com/O=Apple Inc./ST=California/C=US
i:/CN=Apple Server Authentication CA/OU=Certification Authority/O=Apple
Inc./C=US
-----BEGIN CERTIFICATE-----
(...) <- omitted the cert for brevity - Apple's server certificate is here,
i.e. one end of the chain
-----END CERTIFICATE-----
1 s:/CN=Apple Server Authentication CA/OU=Certification Authority/O=Apple
Inc./C=US
i:/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Root CA
-----BEGIN CERTIFICATE-----
(...) <- omitted the cert for brevity - an intermediate cert is here, no
need to worry about that
-----END CERTIFICATE-----
2 s:/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Root CA
i:/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Root CA
-----BEGIN CERTIFICATE-----
(...) HERE you have the Apple's Root CA certificate that you want to copy
and paste, together with the begin/end lines into a new file and convert
that into DER
openssl x509 -outform DER < appleca1.pem > appleca1.der
Repeat that for each server. This will take a couple of hours or even more
than a day ;)
Now having all "der" files you run
gen_certs . siteca.zeek
Now you can copy your new script (the siteca.zeek or whatever you want to
call that) to your "site" directory and @load siteca.zeek it from the
site/local.zeek file
That should take care of all these "private" and OS-specific services.
There's one more thing...
Johanna found out that Zeek by default won't correctly chain up
certificates if the server is not configured correctly. By correctly I mean
- any server should send a full certificate chain, but some don't and we
end up in a situation where all Zeek can see is the root CA certificate
(from its database), the server's certificate not none of the intermediate
certificates.
Browsers deal with it by caching a lot of intermediate certificates and
then doing insane heuristics trying to verify. Well, they have been doing
it for years and for billions of people ;)
I would advise to first deal with SSL::Invalid_Server_Cert by finding out
who generates the "unable to get local issuer certificate" first, i.e. you
want to find the root CA cert and convert it and add to Zeek.
Next, you can decide what do you want to do with self-signed certificates.
You could add them in the same way, or alert on them, depending on your
organization policy.
>
> Also, I put "@/path/to/cacert.zeek" file in my
> "/opt/bro/share/bro/site/local.bro" file. I assume that's where the
> statement goes?
>
> Sorry for all the questions, but I've not found this information
> elsewhere. It really is appreciated.
>
> Jay Fluharty
> Network Analyst
> NS Wheeling-Nisshin Inc.
> PO Box 635
> Follansbee, WV 26037
> jayf at wheeling-nisshin.com
> 1-304-527-4819
>
> [image: Inactive hide details for Michał Purzyński ---11/17/2019 05:45:24
> AM---Excellent question. The reason you see those errors is]Michał
> Purzyński ---11/17/2019 05:45:24 AM---Excellent question. The reason you
> see those errors is the lack of the Root CA in Zeek's
>
> From: Michał Purzyński <michalpurzynski1 at gmail.com>
> To: jayf at wheeling-nisshin.com
> Cc: zeek <zeek at zeek.org>
> Date: 11/17/2019 05:45 AM
> Subject: Re: [Zeek] Certificate questions
> ------------------------------
>
>
>
> Excellent question.
>
> The reason you see those errors is the lack of the Root CA in Zeek's
> certificate store.
>
> Zeek, by default, uses Mozilla certificate store - the same one your
> Firefox uses. Try going to one of these pages, like
> *https://slscr.update.microsoft.com* <https://slscr.update.microsoft.com/> in
> FF and you will see certificate errors as well. You will not see them in
> Edge. Why's that?
>
> For Microsoft, those certificates chain to a CA that has the root CA
> certificate present in the windows certificate store, but nowhere else. For
> Apple, the situation is similar - these root CA certificates are present on
> the system level but no where else.
>
> Since those are for services not accessed by general public, but things
> like iCloud and software updates, these have never been submitted to us for
> inclusion into Mozilla root CA program - and hence never landed in Zeek's
> land.
>
> An example right here here
>
> subject
> CN=*slscr.update.microsoft.com* <http://slscr.update.microsoft.com/>
> ,OU=DSP,O=Microsoft,L=Redmond,ST=WA,C=US
>
> issued by
> CN=Microsoft ECC Update Secure Server CA 2.1,O=Microsoft
> Corporation,L=Redmond,ST=Washington,C=US
>
> issued by
> CN=Microsoft ECC Product Root Certificate Authority 2018,O=Microsoft
> Corporation,L=Redmond,ST=Washington,C=US
>
> Present in MS root store
>
> There is a fix for that - you have to fetch those certificates with tools
> like openssl or the latest Firefox (it's got this nice thing where you can
> download the full chain), transform them into Zeek's scripts and include.
>
> I think Justin wrote a nice script for that.
>
> *https://gist.github.com/JustinAzoff/7a1b92c976a2fa6e8601*
> <https://gist.github.com/JustinAzoff/7a1b92c976a2fa6e8601>
>
> mkdir tmp && cd tmp
>
> openssl s_client -host *slscr.update.microsoft.com*
> <http://slscr.update.microsoft.com/> -port 443 -showcerts < /dev/null |
> sed -n '/BEGIN/,/END/p' | openssl x509 -outform DER > o.der
>
> (do that for each CA - ignore the "verify error")
>
> python ../gen_certs.py . cacert.zeek
>
> And then you can @load the cacert.zeek in a script or in a local.zeek
>
> On Fri, Nov 15, 2019 at 12:37 PM <*jayf at wheeling-nisshin.com*
> <jayf at wheeling-nisshin.com>> wrote:
>
> Greetings Zeek community,
>
> I'm very new to Zeek, but really like what I'm see so far. I need
> some help or perhaps a bit of education though. I have it setup in a
> Security Onion VM.
>
> I see a lot of messages about SSL including "unable to get local
> issuer certificate", which I understand COULD be self-signed certs.
>
> I also see many, many SSL::Invalid_Server_Cert notices in Kibana.
> Many others say "SSL certificate validation failed with (self signed
> certificate in certificate chain).
>
> These would all be of interest, however they ALL point back to very
> legitimate sources like Apple and Microsoft. I find it hard to believe
> that these major companies have problems with that many certificates and
> servers. Could this really be the case???
>
> I could find very little information on Google regarding this. One
> article said something about Zeek not being able to match them up with root
> cert servers or something like that.
>
> Is it possible that Zeek is missing something like a list of root CAs
> or something? Is this just garbage caused by something else. This will
> leave me scratching my head until I come back on Monday. I appreciate the
> help.
>
> Jay Fluharty
> Network Analyst
> NS Wheeling-Nisshin Inc.
> PO Box 635
> Follansbee, WV 26037
> *jayf at wheeling-nisshin.com* <jayf at wheeling-nisshin.com>
> 1-304-527-4819
> _______________________________________________
> Zeek mailing list
> *zeek at zeek.org* <zeek at zeek.org>
> *http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek*
> <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
>
>
>
> _____________________________________________________________________________
> Scanned by IBM Email Security Management Services powered by
> Symantec.Cloud. For more information please visit
> http://www-935.ibm.com/services/us/index.wss/offerfamily/iss/a1026954
>
> _____________________________________________________________________________
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191118/0dd76aed/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191118/0dd76aed/attachment.gif
More information about the Zeek
mailing list