1

I am currently editing a fork of Bitcoin Core. I have three libraries I would like to add as dependencies. One is my own creation (CPISync) and it depends on a second library (NTL) which in turn depends on a third (GMP).

I have looked here and here for ways to add libraries to Bitcoin Core, but neither has proved helpful. Like the first post, I am still seeing Undefined reference to errors every time (and I have tried every permutation to Makefile.am discussed there). Likwise, I have added ntl.mk and gmp.mk files (and successfully run make) like the second advises, but to no avail.

My question is: what is the standard procedure for adding libraries to Bitcoin Core? I mean this both specifically to my case and generally for Bitcoin Core, since my many searches this past week have not revealed any semblance of a beaten path.

My error looks like this:

libbitcoin_server.a(libbitcoin_server_a-net_processing.o): In function `ProcessMessage(CNode*, std::__cxx11::basic_string, std::allocator > const&, CDataStream&, long, CChainParams const&, CConnman*, std::atomic const&)':
/home/hummus/bitcoin/src/net_processing.cpp:1858: undefined reference to `IBLTSync::IBLTSync(unsigned long, unsigned long)'
/home/hummus/bitcoin/src/net_processing.cpp:1863: undefined reference to `DataObject::DataObject(std::__cxx11::basic_string, std::allocator >)'
/home/hummus/bitcoin/src/net_processing.cpp:1864: undefined reference to `IBLTSync::addElem(DataObject*)'
/home/hummus/bitcoin/src/net_processing.cpp:1875: undefined reference to `CommString::CommString(std::__cxx11::basic_string, std::allocator >, bool)'
/home/hummus/bitcoin/src/net_processing.cpp:1879: undefined reference to `IBLTSync::SyncServer(Communicant*, std::__cxx11::list >&, std::__cxx11::list >&)'
/home/hummus/bitcoin/src/net_processing.cpp:1886: undefined reference to `DataObject::to_string[abi:cxx11]() const'
/home/hummus/bitcoin/src/net_processing.cpp:1875: undefined reference to `CommString::~CommString()'
/home/hummus/bitcoin/src/net_processing.cpp:1858: undefined reference to `IBLTSync::~IBLTSync()'
libbitcoin_server.a(libbitcoin_server_a-net_processing.o): In function `ProcessMessage(CNode*, std::__cxx11::basic_string, std::allocator > const&, CDataStream&, long, CChainParams const&, CConnman*, std::atomic const&)':
/usr/include/c++/7/bits/list.tcc:70: undefined reference to `CommString::~CommString()'
/usr/include/c++/7/bits/list.tcc:70: undefined reference to `IBLTSync::~IBLTSync()'
libbitcoin_server.a(libbitcoin_server_a-net_processing.o): In function `ProcessMessage(CNode*, std::__cxx11::basic_string, std::allocator > const&, CDataStream&, long, CChainParams const&, CConnman*, std::atomic const&)':
/home/hummus/bitcoin/src/net_processing.cpp:1858: undefined reference to `IBLTSync::~IBLTSync()'
/home/hummus/bitcoin/src/net_processing.cpp:1875: undefined reference to `CommString::~CommString()'
libntl.a(lip.o): In function `redc(_ntl_gbigint_body*, _ntl_gbigint_body*, long, unsigned long, _ntl_gbigint_body*)':
(.text+0x2a3): undefined reference to `__gmpn_addmul_1'
(.text+0x368): undefined reference to `__gmpn_sub_n'
libntl.a(lip.o): In function `_ntl_rem_struct_basic::eval(long*, _ntl_gbigint_body*, _ntl_tmp_vec*)':
(.text+0x46b): undefined reference to `__gmpn_mod_1'
libntl.a(lip.o): In function `_ntl_reduce_struct_montgomery::eval(_ntl_gbigint_body**, _ntl_gbigint_body**)':
(.text+0xfcb): undefined reference to `__gmpn_addmul_1'
(.text+0x1096): undefined reference to `__gmpn_sub_n'
libntl.a(lip.o): In function `_ntl_crt_struct_basic::eval(_ntl_gbigint_body**, long const*, _ntl_tmp_vec*)':
(.text+0x11df): undefined reference to `__gmpn_addmul_1'
libntl.a(lip.o): In function `gmod_simple(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x1c56): undefined reference to `__gmpn_tdiv_qr'
libntl.a(lip.o): In function `_ntl_rem_struct_fast::eval(long*, _ntl_gbigint_body*, _ntl_tmp_vec*)':
(.text+0x1f5d): undefined reference to `__gmpn_mod_1'
libntl.a(lip.o): In function `_ntl_rem_struct_medium::eval(long*, _ntl_gbigint_body*, _ntl_tmp_vec*)':
(.text+0x21bd): undefined reference to `__gmpn_mod_1'
libntl.a(lip.o): In function `_ntl_grshift(_ntl_gbigint_body*, long, _ntl_gbigint_body**)':
(.text+0x31e1): undefined reference to `__gmpn_rshift'
libntl.a(lip.o): In function `_ntl_glshift(_ntl_gbigint_body*, long, _ntl_gbigint_body**)':
(.text+0x34a1): undefined reference to `__gmpn_lshift'
libntl.a(lip.o): In function `_ntl_gadd(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x3a12): undefined reference to `__gmpn_add_n'
(.text+0x3ab7): undefined reference to `__gmpn_sub_n'
(.text+0x3bda): undefined reference to `__gmpn_sub_n'
(.text+0x3c47): undefined reference to `__gmpn_sub_n'
libntl.a(lip.o): In function `_ntl_gsub(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x3f13): undefined reference to `__gmpn_add_n'
(.text+0x40f7): undefined reference to `__gmpn_sub_n'
(.text+0x41da): undefined reference to `__gmpn_sub_n'
(.text+0x4247): undefined reference to `__gmpn_sub_n'
libntl.a(lip.o): In function `_ntl_gsubpos(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x439c): undefined reference to `__gmpn_sub_n'
libntl.a(lip.o): In function `_ntl_gmul(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x46ba): undefined reference to `__gmpn_mul'
(.text+0x475d): undefined reference to `__gmpn_mul'
libntl.a(lip.o): In function `_ntl_gsmul(_ntl_gbigint_body*, long, _ntl_gbigint_body**)':
(.text+0x48ba): undefined reference to `__gmpn_mul_1'
(.text+0x48f4): undefined reference to `__gmpn_lshift'
libntl.a(lip.o): In function `_ntl_gsdiv(_ntl_gbigint_body*, long, _ntl_gbigint_body**)':
(.text+0x49fd): undefined reference to `__gmpn_divrem_1'
(.text+0x4ab5): undefined reference to `__gmpn_rshift'
libntl.a(lip.o): In function `_ntl_gsmod(_ntl_gbigint_body*, long)':
(.text+0x4ba6): undefined reference to `__gmpn_mod_1'
libntl.a(lip.o): In function `_ntl_gdiv(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**, _ntl_gbigint_body**)':
(.text+0x5052): undefined reference to `__gmpn_tdiv_qr'
libntl.a(lip.o): In function `_ntl_crt_struct_fast::eval(_ntl_gbigint_body**, long const*, _ntl_tmp_vec*)':
(.text+0x55c7): undefined reference to `__gmpn_addmul_1'
libntl.a(lip.o): In function `_ntl_gsqrts(long)':
(.text+0x5856): undefined reference to `__gmpn_sqrtrem'
libntl.a(lip.o): In function `_ntl_gsqrt(_ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x5928): undefined reference to `__gmpn_sqrtrem'
libntl.a(lip.o): In function `_ntl_ggcd(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x5e00): undefined reference to `__gmpn_gcd'
(.text+0x5ec2): undefined reference to `__gmpn_gcd'
libntl.a(lip.o): In function `_ntl_gexteucl(_ntl_gbigint_body*, _ntl_gbigint_body**, _ntl_gbigint_body*, _ntl_gbigint_body**, _ntl_gbigint_body**)':
(.text+0x622a): undefined reference to `__gmpn_gcdext'
libntl.a(lip.o): In function `_ntl_ginv(_ntl_gbigint_body*, _ntl_gbigint_body*, _ntl_gbigint_body**)':
(.text+0x6b64): undefined reference to `__gmpn_gcdext'
libntl.a(lip.o): In function `_ntl_gaorsmul_1(_ntl_gbigint_body*, long, long, _ntl_gbigint_body**)':
(.text+0xa181): undefined reference to `__gmpn_submul_1'
(.text+0xa2fe): undefined reference to `__gmpn_addmul_1'
(.text+0xa43f): undefined reference to `__gmpn_mul_1'
(.text+0xa599): undefined reference to `__gmpn_mul_1'
libntl.a(lip.o): In function `_ntl_quick_accum_muladd(_ntl_gbigint_body*, _ntl_gbigint_body*, long)':
(.text+0xc02b): undefined reference to `__gmpn_addmul_1'
collect2: error: ld returned 1 exit status
Makefile:3643: recipe for target 'bitcoind' failed
make[2]: \*** [bitcoind] Error 1
make[2]: Leaving directory '/home/hummus/bitcoin/src'
Makefile:9297: recipe for target 'all-recursive' failed
make[1]: \*** [all-recursive] Error 1
make[1]: Leaving directory '/home/hummus/bitcoin/src'
Makefile:746: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1
yairks
  • 11
  • 3
  • Have you included the header files correctly (i.e. `#include something.h`) in your code? – Andrew Chow Jul 30 '18 at 15:50
  • If they are libraries (static or dynamic libs), then you also have to add the .a or .so files to the linker script in the makefiles – JBaczuk Jul 30 '18 at 16:00
  • @AndrewChow I believe I did. – yairks Jul 30 '18 at 16:05
  • @JBaczuk I did that as well. `LIBCPISYNC=CPISync.a`, etc. I added these libraries to various parts of Makefile.am and came up dry each time. – yairks Jul 30 '18 at 16:06
  • Can you post your Makefile.am? There are some ordering things in makefiles that makes adding libraries annoying. In general, the orderings will be different depending on where the libraries you are adding are being used. – Andrew Chow Jul 30 '18 at 16:09
  • @AndrewChow Here's my current Makefile (although it has changed a bit during my tribulations): https://drive.google.com/open?id=1DkRssXfPM0eNqvYVa3B4oiMNKF7dY1_N – yairks Jul 30 '18 at 16:14
  • Where are the files `libcpisynclib.a`, `libntl.a`, and `ibgmp.a` located? They aren't being compiled by the Makefile. If you are expecting them to be compiled by the makefile, then you need to add targets for those libraries. For how to do that, look at how it is done for `LIBSECP256K1`. Otherwise, if you have compiled those libraries yourself, then they need to be in `src/`. – Andrew Chow Jul 30 '18 at 16:26
  • @AndrewChow Alas, that too I have already tried. All three (compiled) files are already located in `src/`. But even doing that felt strange: why should I have to move *.a files from `usr/lib` to `src`? Can't Bitcoin Core find them in their normal locations? – yairks Jul 30 '18 at 16:34
  • Have you tried not adding your library's source code to the compilation? i.e. just use the library when it is linked, instead of compiling it into bitcoind. It seems like that would cause problems. – Andrew Chow Jul 30 '18 at 16:47
  • Can you post your source code so that I can try to compile it? Make is one of those things that usually needs to be done a case by case basis and there isn't really a generic answer that can be given without building the code itself. – Andrew Chow Jul 30 '18 at 16:50
  • @AndrewChow I do that as well in the current Makefile. If you would like to try and run it yourself (which would be very sweet), you can find the code [here](http://github.com/Yairks/bitcoin). – yairks Jul 30 '18 at 16:55
  • 1
    I see you are trying to link cpisync into Bitcoin, you may be interested in Minisketch ( https://github.com/sipa/minisketch/ ) which is depdendencyless, compatibly licensed, and much faster. – G. Maxwell Jan 07 '19 at 04:30

1 Answers1

1

I was able to get the code to compile with the following diff:

diff --git a/src/Makefile.am b/src/Makefile.am
index 6bb19d9bf..aab8ab458 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -33,9 +33,8 @@ LIBBITCOIN_CRYPTO=crypto/libbitcoin_crypto.a
 LIBBITCOINQT=qt/libbitcoinqt.a
 LIBSECP256K1=secp256k1/libsecp256k1.la
 #My personal libaries
-LIBCPISYNCLIB=libcpisynclib.a
-LIBNTL=libntl.a
-LIBGMP=libgmp.a
+LIBNTL=-lntl
+LIBGMP=-lgmp


 if ENABLE_ZMQ
@@ -442,7 +441,7 @@ bitcoind_LDADD = \
   $(LIBMEMENV) \
   $(LIBSECP256K1)

-bitcoind_LDADD += $(BOOST_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(ZMQ_LIBS) $(LIBCPISYNCLIB) $(LIBNTL) $(LIBGMP)
+bitcoind_LDADD += $(BOOST_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(ZMQ_LIBS) $(LIBNTL) $(LIBGMP)

 # bitcoin-cli binary #
 bitcoin_cli_SOURCES = bitcoin-cli.cpp
@@ -460,7 +459,7 @@ bitcoin_cli_LDADD = \
   $(LIBBITCOIN_UTIL) \
   $(LIBBITCOIN_CRYPTO)

-bitcoin_cli_LDADD += $(BOOST_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(EVENT_LIBS) $(LIBCPISYNCLIB) $(LIBNTL) $(LIBGMP)
+bitcoin_cli_LDADD += $(BOOST_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(EVENT_LIBS) $(LIBNTL) $(LIBGMP)
 #

 # bitcoin-tx binary #
@@ -481,7 +480,7 @@ bitcoin_tx_LDADD = \
   $(LIBBITCOIN_CRYPTO) \
   $(LIBSECP256K1)

-bitcoin_tx_LDADD += $(BOOST_LIBS) $(CRYPTO_LIBS) $(LIBCPISYNCLIB) $(LIBNTL) $(LIBGMP)
+bitcoin_tx_LDADD += $(BOOST_LIBS) $(CRYPTO_LIBS) $(LIBNTL) $(LIBGMP)
 #

 # bitcoinconsensus library #
diff --git a/src/Makefile.bench.include b/src/Makefile.bench.include
index 2b1f70b25..796348338 100644
--- a/src/Makefile.bench.include
+++ b/src/Makefile.bench.include
@@ -45,7 +45,7 @@ bench_bench_bitcoin_LDADD = \
   $(LIBUNIVALUE)

 if ENABLE_ZMQ
-bench_bench_bitcoin_LDADD += $(LIBBITCOIN_ZMQ) $(ZMQ_LIBS)
+bench_bench_bitcoin_LDADD += $(LIBBITCOIN_ZMQ) $(ZMQ_LIBS) $(LIBNTL) $(LIBGMP)
 endif

 if ENABLE_WALLET
diff --git a/src/Makefile.qt.include b/src/Makefile.qt.include
index 7b44ce038..59e250539 100644
--- a/src/Makefile.qt.include
+++ b/src/Makefile.qt.include
@@ -424,7 +424,7 @@ qt_bitcoin_qt_LDADD += $(LIBBITCOIN_ZMQ) $(ZMQ_LIBS)
 endif
 qt_bitcoin_qt_LDADD += $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CONSENSUS) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) $(LIBLEVELDB) $(LIBLEVELDB_SSE42) $(LIBMEMENV) \
   $(BOOST_LIBS) $(QT_LIBS) $(QT_DBUS_LIBS) $(QR_LIBS) $(PROTOBUF_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(LIBSECP256K1) \
-  $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS)
+  $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(LIBNTL) $(LIBGMP)
 qt_bitcoin_qt_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(QT_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
 qt_bitcoin_qt_LIBTOOLFLAGS = --tag CXX

diff --git a/src/Makefile.qttest.include b/src/Makefile.qttest.include
index ea2ed1747..a322fbf39 100644
--- a/src/Makefile.qttest.include
+++ b/src/Makefile.qttest.include
@@ -60,7 +60,7 @@ endif
 qt_test_test_bitcoin_qt_LDADD += $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CONSENSUS) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) $(LIBLEVELDB) \
   $(LIBLEVELDB_SSE42) $(LIBMEMENV) $(BOOST_LIBS) $(QT_DBUS_LIBS) $(QT_TEST_LIBS) $(QT_LIBS) \
   $(QR_LIBS) $(PROTOBUF_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(LIBSECP256K1) \
-  $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS)
+  $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(LIBNTL) $(LIBGMP)
 qt_test_test_bitcoin_qt_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(QT_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
 qt_test_test_bitcoin_qt_CXXFLAGS = $(AM_CXXFLAGS) $(QT_PIE_FLAGS)

diff --git a/src/Makefile.test.include b/src/Makefile.test.include
index b48bfc76b..0ac10dece 100644
--- a/src/Makefile.test.include
+++ b/src/Makefile.test.include
@@ -103,7 +103,7 @@ test_test_bitcoin_LDADD += $(LIBBITCOIN_SERVER) $(LIBBITCOIN_CLI) $(LIBBITCOIN_C
   $(LIBLEVELDB) $(LIBLEVELDB_SSE42) $(LIBMEMENV) $(BOOST_LIBS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(LIBSECP256K1) $(EVENT_LIBS) $(EVENT_PTHREADS_LIBS)
 test_test_bitcoin_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)

-test_test_bitcoin_LDADD += $(LIBBITCOIN_CONSENSUS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS)
+test_test_bitcoin_LDADD += $(LIBBITCOIN_CONSENSUS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(LIBNTL) $(LIBGMP)
 test_test_bitcoin_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) -static

 if ENABLE_ZMQ

There are a few important things to note here. First, I removed LIBCPISYNCLIB. This is unnecessary because the source code for the library is already being compiled into the binary.

Secondly, I changed LIBNTL and LIBGMP to -lntl and -lgmp respectively. This tells the linker to look for the system installed libraries instead of specific .a files in the source code.

Lastly, $(LIBNTL) $(LIBGMP) was added to the other Makefile.am.*.include files. These libraries must be added wherever libbitcoin_server is used as there are more than just the bitcoind, bitcoin-cli, and bitcoin-tx binaries built. There are binaries for tests, benchmarks, bitcoin-qt, and bitcoin-qt tests.

Andrew Chow
  • 67,209
  • 5
  • 76
  • 149
  • Hmm... That code does look logical, but when I changed the files as you instructed I got the same error as before (although now the undefined references are for NTL). Just to be sure, I downloaded my own source code a second time and got the same problem. Any other suggestions? – yairks Jul 30 '18 at 19:20
  • You need to install NTL to your system. I suggest recompiling it and then doing `sudo make install` – Andrew Chow Jul 30 '18 at 19:36