OpenLDAP with TLS AD backend

Question

I'm currently using OpenLDAP as a read-only proxy for an internal Active Directory domain. My main issue is the connection works fine between AD DC and proxy without SSL, it doesn't work with it.

My configuration is as follows :

include                 /etc/openldap/schema/core.schema
include                 /etc/openldap/schema/cosine.schema
include                 /etc/openldap/schema/nis.schema
include                 /etc/openldap/schema/inetorgperson.schema
include                 /etc/openldap/schema/misc.schema

loglevel                none
modulepath              /usr/lib64/openldap/

pidfile                 /var/run/openldap/slapd.pid

moduleload              back_ldap.la

database                ldap
readonly                yes
uri                     "ldap://ad-fqdn"
suffix                  "suffix"
binddn                  "dn"
bindpw                  "password"

TLSCACertificateFile    /usr/local/etc/openldap/torque.pem
TLSCertificateFile      /usr/local/etc/openldap/ad-proxy.pem
TLSCertificateKeyFile   /usr/local/etc/openldap/ad-proxy.key

If I change "ldap" to "ldaps" in the URI, it doesn't work anymore. The CA is the same between my AD DC and my proxy, and I can query the AD DC with ldapsearch from proxy (with TLS) without any issue. The CA is trusted on my ad-proxy (CentOS 7) machine.

How can I use SSL over my connection ?

Thanks for your help.

EDIT: Nevermind, I figured it out. Cert was not valid for FQDN.