.NET Core makes it convenient to develop and test C# code across platforms. On my current project, this means we can do much of our work on Macs without ever firing up a Windows VM.
Even the best abstraction layers occasionally leak, though. Here’s a story of an OSX-specific issue we encountered, what we learned, and how to resolve it.
## The Problem
So there I was, running an existing .NET Framework project on .NET Core for the first time (using .NET Core 1.1). The project uses a third-party library to communicate with a REST API. Like many problems in software development, it started with an error message:
System.PlatformNotSupportedException: The libcurl library in use (7.54.0) and its SSL backend ("SecureTransport") do not support custom handling of certificates. A libcurl built with OpenSSL is required.
libcurl provides an HTTP client, and it can be built with one of [several][curl_backends] TLS backends. It looked like we needed to be using OpenSSL instead of SecureTransport (which is Apple’s [TLS Implementation][secure_transport]).
For us, the exception came from within a third-party library, but you might encounter it if you were doing something advanced with [HttpClient][httpclient_docs]. For example, a contributor on the GitHub issue provided a short [repro][github_issue] that attempts to check a certificate revocation list.
## Get libcurl
So how can we get libcurl built with OpenSSL? The easiest way is to install Homebrew’s `curl` [package][homebrew_curl_package], which provides both the `curl` command line tool and the `libcurl` library. It, too, uses SecureTransport by default, but we can choose OpenSSL instead:
`brew install curl –with-openssl`
After it’s installed, you’ll get a warning:
This formula is keg-only, which means it was not symlinked into /usr/local, because macOS already provides this software and installing another version in parallel can cause all kinds of trouble.
Let’s see what kind of trouble, shall we?
## A Bad Solution
One suggestion on the issue thread is to disregard the warning and force-link libcurl (`brew link –force curl`). This would promote `curl` and `libcurl` to paths within `/usr/local`. Then feed the library path to the loader by setting an environment variable: `DYLD_LIBRARY_PATH=/usr/local/lib`.
_Side note: `DYLD_LIBRARY_PATH` is OSX’s version of `LD_PRELOAD`, which is [awesome][ld_preload_twitter], and you can read more about it [here][ld_preload_jvns]._
With this environment variable in place, when the loader goes looking for libraries, it will look in `/usr/local/lib` first. This fixes our libcurl issue by prioritizing Brew’s version over the system version. Sweet!
But what if there were some other libraries in there that we didn’t want? Like, what if we needed Apple’s special version of libJPEG because it provides extra symbols that Homebrew’s libJPEG doesn’t offer?
Failed to load /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.1.2/libcoreclr.dylib, error: dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.1.2/libcoreclr.dylib, 1): Symbol not found: __cg_jpeg_resync_to_restart Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO Expected in: /usr/local/lib/libJPEG.dylib in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO Failed to bind to CoreCLR at '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.1.2/libcoreclr.dylib'
Oops. This also happens with libtiff (`__cg_TIFFClientOpen`) and libpng (`__cg_png_create_info_struct`). For a practical repro of the conflict, `brew install imagemagick`, which depends on all three. If you ever need to analyze problems like this more thoroughly, you can view library and symbol dependencies with the `otool` and `nm` commands.
To fix _this_, we could uninstall or unlink those libs, but that feels like setting another hack on a leaning pile of hacks.
## A Better Solution
Instead of prioritizing _all_ of Homebrew’s libraries, it’s safer to take just what we need: `DYLD_LIBRARY_PATH=/usr/local/opt/curl/lib`. No force link necessary. Tada!
You could also use this technique to provide Homebrew’s OpenSSL to dotnet, instead of manually symlinking the libs as the current installation guide [instructs][netcore_getting_started].
## The Best Solution
In the future, this won’t be necessary at all, as .NET Core 2.0 [won’t rely on OpenSSL on OSX](https://github.com/dotnet/announcements/issues/21). If you install the [2.0 preview](https://www.microsoft.com/net/core/preview#macos) and target e.g. `netcoreapp2.0`, the problem goes away!
If you’re stuck on 1.x, the _Better Solution_ above offers a feasible workaround that doesn’t require you to ignore your package manager’s warnings and introduce risk to other software on your system. If you’re able, though, now’s a good time to try out .NET Core 2.0. It should be releasing [very soon][netcore_2_milestone]!
_Update: since this post was written, .NET Core 2.0 has been [released][netcore_2_release]!_
I was going along this path but was using the Linux environment variables by mistake, thanks for clearing this up! I wanted to point out that if you’re using self signed certificates on a server for development, you may still need this even in .NET Core 2.0.
Here’s my scenario:
I’m building a web api that is relying on IdentityServer4 to issue tokens for user authorization. After getting the token server to work, I secured one of my api endpoints (I had to specifically specify the authentication scheme in the Authorize attribute) all was working fine! Until I tried to use my self-signed cert to secure communications (this will change in production of course). The authentication pipeline for the token makes a web request to the token authority. Within that code, a HttpClient object is being created and used. By default, the HttpClient will use libcurl which will not allow self-signed certs even if you have added them to your system keychain. There was a workaround that was added to .NET Core 2.0 for this which was to set your custom callback validator to a specific statically exposed delegate on the HttpClientHandler.
handler.ServerCertificateCustomValidationCallback = System.Net.Http.HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
This does solve the problem if you have access to the HttpClient object and its handler. In my case, that is buried deep in the authentication handler pipeline of .NET Core. Thus, I had to use this workaround.
Thanks for the great post!
Thanks for the comment, Dave! I’m glad the post was helpful. How has your experience been with Identity Server 4?
So far so good. I’m still trying to figure out the best way to handle non-web interactions with IdentityServer 4. We’ll see how it turns out. My goal is to have my own backend that supports asp.net identity, but also allow users to use FB or other oauth2 compatible sites to login as well.
I recently had my system crash due to bad memory and now self signed cert is not cooperating the way it was. I wish macOS libraries and utilities would trust the self signed certs the same way it does any other. It just doesn’t want to do that.
I’m still having this issue with .NET Core 2.
Did you manage to solve that on Mac?
In hindsight, I was lucky that my specific issue was mitigated with .NET Core 2. Between you and Dave, it’s clear that platform-specific crypto problems remain!
Hopefully they resolve these in future releases, but in the short term it might be worth looking into using another framework implementation like Mono, or maybe .NET Core in a Linux container.
Comments are closed.