TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, required_outer))", File: "../../../../../../../src/postgres/src/backend/optimizer/util/relnode.c", Line: 1543) 2022-02-03 08:13:00.054 UTC [22132] LOG: server process (PID 18763) was terminated by signal 6: Aborted 2022-02-03 08:13:00.054 UTC [22132] DETAIL: Failed process was running: select subq_0.c0 as c0, subq_0.c0 as c1, subq_0.c0 as c2, (select authorization_identifier from information_schema._pg_foreign_servers limit 1 offset 5) as c3	from (select (select opcowner from pg_catalog.pg_opclass limit 1 offset 5) as c0 from public.table_create_ctas_nodata as ref_0 where cast(null as lseg) ?# pg_catalog.line( cast(cast(null as point) as point), cast(cast(null as point) as point))) as subq_0	where EXISTS ( select subq_3.c1 as c0, subq_2.c1 as c1, subq_2.c1 as c2, pg_catalog.inet_server_port() as c3, subq_1.c0 as c4 from (select subq_0.c0 as c0 from information_schema.column_domain_usage as ref_1 where (select prosrc from pg_catalog.pg_proc limit 1 offset 6) ~>=~ (select pg_catalog.max(application_name) from pg_catalog.pg_stat_activity) limit 73) as subq_1, lateral (sel 2022-02-03 08:13:00.054 UTC [22132] LOG: terminating any other active server processes 2022-02-03 08:13:00.054 UTC [18766] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.054 UTC [18766] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.054 UTC [18766] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.055 UTC [18761] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.055 UTC [18761] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.055 UTC [18761] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.055 UTC [18760] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.055 UTC [18760] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.055 UTC [18760] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.055 UTC [18768] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.055 UTC [18768] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.055 UTC [18768] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.056 UTC [18767] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.056 UTC [18767] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.056 UTC [18767] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.056 UTC [18765] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.056 UTC [18765] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.056 UTC [18765] HINT: In a moment you should be able to reconnect to the database and repeat your command. 2022-02-03 08:13:00.056 UTC [18762] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.056 UTC [18762] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.056 UTC [18762] HINT: In a moment you should be able to reconnect to the database and repeat your command. I0203 08:13:00.062014 18817 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) I0203 08:13:00.067651 18770 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) I0203 08:13:00.070192 18805 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) I0203 08:13:00.070571 18780 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) 2022-02-03 08:13:00.071 UTC [18764] WARNING: terminating connection because of crash of another server process 2022-02-03 08:13:00.071 UTC [18764] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2022-02-03 08:13:00.071 UTC [18764] HINT: In a moment you should be able to reconnect to the database and repeat your command. I0203 08:13:00.077311 18803 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) W0203 08:13:00.078380 18808 outbound_call.cc:131] Failed to schedule invoking callback on response for request yb.tserver.TabletServerService.Read to 127.0.0.1: Aborted (yb/rpc/thread_pool.cc:260): Service is shutting down I0203 08:13:00.082116 18793 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) I0203 08:13:00.088299 18838 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) I0203 08:13:00.089371 18832 poller.cc:66] Poll stopped: Service unavailable (yb/rpc/scheduler.cc:80): Scheduler is shutting down (system error 108) W0203 08:13:00.113699 18791 thread.cc:781] Thread join failed: Invalid argument (yb/util/thread.cc:611): Can't join on own thread: iotp_pggate_ybclient_0 W0203 08:13:00.121057 18791 thread.cc:781] Thread join failed: Invalid argument (yb/util/thread.cc:611): Can't join on own thread: iotp_pggate_ybclient_0 2022-02-03 08:13:02.150 UTC [18929] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.295 UTC [18931] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.295 UTC [18930] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.305 UTC [18932] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.306 UTC [18933] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.441 UTC [18934] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.441 UTC [18935] FATAL: the database system is in recovery mode 2022-02-03 08:13:02.442 UTC [18936] FATAL: the database system is in recovery mode ================================================================= ==18764==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000025010 at pc 0x7f463a78d766 bp 0x7ffdcc6757a0 sp 0x7ffdcc675798 READ of size 8 at 0x602000025010 thread T0 #0 0x7f463a78d765 in std::__1::unique_ptr<yb::client::YBClient::Data, std::__1::default_delete<yb::client::YBClient::Data> >::operator->() const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/memory:1653:19 #1 0x7f463a783c3e in yb::client::internal::Batcher::FlushAsync(boost::function<void (yb::Status const&)>, yb::StronglyTypedBool<yb::client::internal::IsWithinTransactionRetry_Tag>) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/batcher.cc:271:7 #2 0x7f463ab6bbf3 in yb::client::(anonymous namespace)::FlushBatcherAsync(std::__1::shared_ptr<yb::client::internal::Batcher> const&, boost::function<void (yb::client::FlushStatus*)>, yb::client::YBSession::BatcherConfig, yb::StronglyTypedBool<yb::client::internal::IsWithinTransactionRetry_Tag>) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/session.cc:210:12 #3 0x7f463ab6b877 in yb::client::YBSession::FlushAsync(boost::function<void (yb::client::FlushStatus*)>) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/session.cc:229:5 #4 0x7f463ab6b45c in yb::client::YBSession::FlushFuture() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/session.cc:241:3 #5 0x7f463fbd3f3d in yb::pggate::PgSession::RunHelper::Flush() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_session.cc:445:39 #6 0x7f463fc60678 in yb::Result<yb::pggate::PgSessionAsyncRunResult> yb::pggate::PgSession::RunAsync<yb::client::YBPgsqlOp>(std::__1::shared_ptr<yb::client::YBPgsqlOp> const*, unsigned long, yb::PgObjectId const&, unsigned long*, bool) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_session.h:251:19 #7 0x7f463fc5750d in yb::pggate::PgDocOp::SendRequestImpl(bool) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_doc_op.cc:283:15 #8 0x7f463fc55c3a in yb::pggate::PgDocOp::SendRequest(bool) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_doc_op.cc:266:18 #9 0x7f463fc56258 in yb::pggate::PgDocOp::GetResult(std::__1::list<yb::pggate::PgDocResult, std::__1::allocator<yb::pggate::PgDocResult> >*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_doc_op.cc:202:22 #10 0x7f463fc15c63 in yb::pggate::PgDml::FetchDataFromServer() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_dml.cc:313:3 #11 0x7f463fc1504a in yb::pggate::PgDml::Fetch(int, unsigned long*, bool*, PgSysColumns*, bool*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pg_dml.cc:301:10 #12 0x7f463fb6dc97 in yb::pggate::PgApiImpl::DmlFetch(yb::pggate::PgStatement*, int, unsigned long*, bool*, PgSysColumns*, bool*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:931:37 #13 0x7f463fb34fea in YBCPgDmlFetch /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:618:29 #14 0xbd0ce9 in ybcIterateForeignScan /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/ybc_fdw.c:529:17 #15 0xbacdff in ForeignNext /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeForeignscan.c:0:0 #16 0xb0b38c in ExecScanFetch /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execScan.c:95:9 #17 0xb0af12 in ExecScan /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execScan.c:171:10 #18 0xbac283 in ExecForeignScan /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeForeignscan.c:126:9 #19 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #20 0xb8f7b9 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #21 0xb8eb23 in ExecNestLoop /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeNestloop.c:109:21 #22 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #23 0xb9b539 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #24 0xb9a417 in ExecSort /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeSort.c:116:11 #25 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #26 0xbbb909 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #27 0xbb8a6e in begin_partition /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeWindowAgg.c:1111:31 #28 0xbaf34e in ExecWindowAgg /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeWindowAgg.c:2104:3 #29 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #30 0xb9b539 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #31 0xb9a417 in ExecSort /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeSort.c:116:11 #32 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #33 0xbbb909 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #34 0xbb8a6e in begin_partition /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeWindowAgg.c:1111:31 #35 0xbaf34e in ExecWindowAgg /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/nodeWindowAgg.c:2104:3 #36 0xb0732c in ExecProcNodeFirst /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execProcnode.c:446:9 #37 0xaf54a9 in ExecProcNode /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/include/executor/executor.h:249:9 #38 0xaebf95 in ExecutePlan /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execMain.c:1730:10 #39 0xaebbbf in standard_ExecutorRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execMain.c:367:3 #40 0x7f462374b524 in pgss_ExecutorRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/contrib/pg_stat_statements/../../../../../src/postgres/contrib/pg_stat_statements/pg_stat_statements.c:947:4 #41 0x7f4623731aaf in ybpgm_ExecutorRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/contrib/yb_pg_metrics/../../../../../src/postgres/contrib/yb_pg_metrics/yb_pg_metrics.c:499:7 #42 0xaeb8b7 in ExecutorRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/executor/../../../../../../src/postgres/src/backend/executor/execMain.c:305:3 #43 0xffb7da in PortalRunSelect /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/pquery.c:955:4 #44 0xffaa63 in PortalRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/pquery.c:784:18 #45 0xff5ad4 in exec_simple_query /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:1161:10 #46 0xff36d8 in yb_exec_simple_query_impl /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:4464:2 #47 0xff3592 in yb_exec_query_wrapper /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:4445:4 #48 0xfed791 in yb_exec_simple_query /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:4479:2 #49 0xfeba42 in PostgresMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:5087:23 #50 0xe38f6c in BackendRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4470:2 #51 0xe37d75 in BackendStartup /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4136:3 #52 0xe35b35 in ServerLoop /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1754:7 #53 0xe3213a in PostmasterMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1417:11 #54 0xc22999 in PostgresServerProcessMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/main/../../../../../../src/postgres/src/backend/main/main.c:234:3 #55 0xc230a1 in main ??:0:0 #56 0x7f463e535554 in __libc_start_main ??:0:0 #57 0x488738 in _start ??:0:0 0x602000025010 is located 0 bytes inside of 8-byte region [0x602000025010,0x602000025018) freed by thread T1 (iotp_pggate_ybc) here: #0 0x7f46433035ad in operator delete(void*) _asan_rtl_:3 #1 0x7f463a74a227 in yb::AtomicUniquePtr<yb::client::YBClient>::~AtomicUniquePtr() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/util/atomic.h:358:5 #2 0x7f463a748fe3 in yb::client::AsyncClientInitialiser::~AsyncClientInitialiser() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/async_initializer.cc:74:1 #3 0x7f463fb65306 in yb::pggate::PgApiImpl::~PgApiImpl() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:271:1 #4 0x7f463fb6543d in yb::pggate::PgApiImpl::~PgApiImpl() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:267:25 #5 0x7f463fb2dba4 in YBCDestroyPgGate /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:132:5 #6 0x143811a in YBOnPostgresBackendShutdown /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/utils/misc/../../../../../../../src/postgres/src/backend/utils/misc/pg_yb_utils.c:513:2 #7 0xfe7c02 in quickdie /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:2683:3 #8 0x7f463e8f062f in _L_unlock_13 funlockfile.c:? #9 0x7f463141a95e in boost::asio::detail::conditionally_enabled_event::wait(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/include/boost/asio/detail/conditionally_enabled_event.hpp:89:14 #10 0x7f4631419343 in boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/include/boost/asio/detail/impl/scheduler.ipp:409:21 #11 0x7f4631418f51 in boost::asio::detail::scheduler::run(boost::system::error_code&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/include/boost/asio/detail/impl/scheduler.ipp:154:10 #12 0x7f4631418bd7 in boost::asio::io_context::run(boost::system::error_code&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/include/boost/asio/impl/io_context.ipp:70:16 #13 0x7f463140f333 in yb::rpc::IoThreadPool::Impl::Execute() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/rpc/io_thread_pool.cc:76:17 #14 0x7f4631417beb in _ZNSt3__18__invokeIRMN2yb3rpc12IoThreadPool4ImplEFvvERPS4_JEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OSA_DpOSB_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/type_traits:3635:1 #15 0x7f4631417b62 in _ZNSt3__115__apply_functorIMN2yb3rpc12IoThreadPool4ImplEFvvENS_5tupleIJPS4_EEEJLm0EENS7_IJEEEEENS_13__bind_returnIT_T0_T2_Xsr22__is_valid_bind_returnISC_SD_SE_EE5valueEE4typeERSC_RSD_NS_15__tuple_indicesIJXspT1_EEEEOSE_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2857:12 #16 0x7f4631417ae3 in _ZNSt3__16__bindIMN2yb3rpc12IoThreadPool4ImplEFvvEJPS4_EEclIJEEENS_13__bind_returnIS6_NS_5tupleIJS7_EEENSB_IJDpOT_EEEXsr22__is_valid_bind_returnIS6_SC_SG_EE5valueEE4typeESF_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2890:20 #17 0x7f4631417a18 in _ZNSt3__18__invokeIRNS_6__bindIMN2yb3rpc12IoThreadPool4ImplEFvvEJPS5_EEEJEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSB_DpOSC_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/type_traits:3694:1 #18 0x7f46314179f0 in _ZNSt3__115__apply_functorINS_6__bindIMN2yb3rpc12IoThreadPool4ImplEFvvEJPS5_EEENS_5tupleIJEEEJESB_EENS_13__bind_returnIT_T0_T2_Xsr22__is_valid_bind_returnISD_SE_SF_EE5valueEE4typeERSD_RSE_NS_15__tuple_indicesIJXspT1_EEEEOSF_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2857:12 #19 0x7f4631417987 in _ZNSt3__16__bindINS0_IMN2yb3rpc12IoThreadPool4ImplEFvvEJPS4_EEEJEEclIJEEENS_13__bind_returnIS8_NS_5tupleIJEEENSC_IJDpOT_EEEXsr22__is_valid_bind_returnIS8_SD_SH_EE5valueEE4typeESG_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2890:20 #20 0x7f46314178b8 in _ZNSt3__18__invokeIRNS_6__bindINS1_IMN2yb3rpc12IoThreadPool4ImplEFvvEJPS5_EEEJEEEJEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSC_DpOSD_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/type_traits:3694:1 #21 0x7f4631417888 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>>&>(std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>>&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/__functional_base:348:9 #22 0x7f4631417860 in std::__1::__function::__alloc_func<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>>, std::__1::allocator<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>> >, void ()>::operator()() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:1558:16 #23 0x7f463141609c in std::__1::__function::__func<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>>, std::__1::allocator<std::__1::__bind<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>> >, void ()>::operator()() /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:1732:12 #24 0x7f4638d814e8 in std::__1::__function::__value_func<void ()>::operator()() const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:1885:16 #25 0x7f4638d7fe68 in std::__1::function<void ()>::operator()() const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2560:12 #26 0x7f462fbcd9eb in yb::Thread::SuperviseThread(void*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/util/thread.cc:774:3 #27 0x7f463e8e8ea4 in start_thread ??:0:0 previously allocated by thread T7 here: #0 0x7f4643302d4d in operator new(unsigned long) _asan_rtl_:3 #1 0x7f463a7b1618 in yb::client::YBClientBuilder::DoBuild(yb::rpc::Messenger*, std::__1::unique_ptr<yb::client::YBClient, std::__1::default_delete<yb::client::YBClient> >*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/client.cc:438:31 #2 0x7f463a7b2cc1 in yb::client::YBClientBuilder::Build(yb::rpc::Messenger*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/client.cc:510:3 #3 0x7f463a7494b6 in yb::client::AsyncClientInitialiser::InitClient() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/async_initializer.cc:89:36 #4 0x7f463a752ceb in _ZNSt3__18__invokeIRMN2yb6client22AsyncClientInitialiserEFvvERPS3_JEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS9_DpOSA_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/type_traits:3635:1 #5 0x7f463a752c62 in _ZNSt3__115__apply_functorIMN2yb6client22AsyncClientInitialiserEFvvENS_5tupleIJPS3_EEEJLm0EENS6_IJEEEEENS_13__bind_returnIT_T0_T2_Xsr22__is_valid_bind_returnISB_SC_SD_EE5valueEE4typeERSB_RSC_NS_15__tuple_indicesIJXspT1_EEEEOSD_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2857:12 #6 0x7f463a752be3 in _ZNSt3__16__bindIMN2yb6client22AsyncClientInitialiserEFvvEJPS3_EEclIJEEENS_13__bind_returnIS5_NS_5tupleIJS6_EEENSA_IJDpOT_EEEXsr22__is_valid_bind_returnIS5_SB_SF_EE5valueEE4typeESE_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/functional:2890:20 #7 0x7f463a752af8 in _ZNSt3__18__invokeINS_6__bindIMN2yb6client22AsyncClientInitialiserEFvvEJPS4_EEEJEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS9_DpOSA_ /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/type_traits:3694:1 #8 0x7f463a752aa0 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::__bind<void (yb::client::AsyncClientInitialiser::*)(), yb::client::AsyncClientInitialiser*>>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::__bind<void (yb::client::AsyncClientInitialiser::*)(), yb::client::AsyncClientInitialiser*>>&, std::__1::__tuple_indices<>) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/thread:280:5 #9 0x7f463a752390 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::__bind<void (yb::client::AsyncClientInitialiser::*)(), yb::client::AsyncClientInitialiser*> > >(void*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/thread:291:5 #10 0x7f463e8e8ea4 in start_thread ??:0:0 Thread T1 (iotp_pggate_ybc) created by T0 here: #0 0x7f46432e113c in pthread_create _asan_rtl_:3 #1 0x7f462fbccb99 in yb::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void ()>, scoped_refptr<yb::Thread>*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/util/thread.cc:696:15 #2 0x7f463140effd in yb::Result<scoped_refptr<yb::Thread> > yb::Thread::Make<std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*> >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::__bind<void (yb::rpc::IoThreadPool::Impl::*)(), yb::rpc::IoThreadPool::Impl*>&&) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/util/thread.h:203:5 #3 0x7f463140e1dd in yb::rpc::IoThreadPool::Impl::Impl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/rpc/io_thread_pool.cc:38:26 #4 0x7f463140dd3e in yb::rpc::IoThreadPool::IoThreadPool(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/rpc/io_thread_pool.cc:87:17 #5 0x7f4631424c0d in yb::rpc::Messenger::Messenger(yb::rpc::MessengerBuilder const&) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/rpc/messenger.cc:552:7 #6 0x7f463141ed93 in yb::rpc::MessengerBuilder::Build() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/rpc/messenger.cc:149:44 #7 0x7f463aab4183 in yb::client::CreateClientMessenger(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, scoped_refptr<yb::MetricEntity> const&, std::__1::shared_ptr<yb::MemTracker> const&, yb::rpc::SecureContext*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/client_utils.cc:68:20 #8 0x7f463fb63ab3 in yb::pggate::(anonymous namespace)::BuildMessenger(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, scoped_refptr<yb::MetricEntity> const&, std::__1::shared_ptr<yb::MemTracker> const&) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:104:20 #9 0x7f463fb633dc in yb::pggate::PgApiContext::PgApiContext() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:202:24 #10 0x7f463fb2d52d in yb::pggate::YBCInitPgGateEx(PgTypeEntity const*, int, PgCallbacks, yb::pggate::PgApiContext*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:115:35 #11 0x7f463fb2d9b0 in YBCInitPgGate /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:123:3 #12 0x1437e54 in YBInitPostgresBackend /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/utils/misc/../../../../../../../src/postgres/src/backend/utils/misc/pg_yb_utils.c:497:3 #13 0x13f014c in InitPostgres /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/utils/init/../../../../../../../src/postgres/src/backend/utils/init/postinit.c:0:3 #14 0xfeaaa5 in PostgresMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:4707:2 #15 0xe38f6c in BackendRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4470:2 #16 0xe37d75 in BackendStartup /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4136:3 #17 0xe35b35 in ServerLoop /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1754:7 #18 0xe3213a in PostmasterMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1417:11 #19 0xc22999 in PostgresServerProcessMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/main/../../../../../../src/postgres/src/backend/main/main.c:234:3 #20 0xc230a1 in main ??:0:0 #21 0x7f463e535554 in __libc_start_main ??:0:0 Thread T7 created by T0 here: #0 0x7f46432e113c in pthread_create _asan_rtl_:3 #1 0x7f463a752282 in std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/__threading_support:509:10 #2 0x7f463a74a6c9 in std::__1::thread::thread<std::__1::__bind<void (yb::client::AsyncClientInitialiser::*)(), yb::client::AsyncClientInitialiser*>, , void>(std::__1::__bind<void (yb::client::AsyncClientInitialiser::*)(), yb::client::AsyncClientInitialiser*>&&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20211222064215-dd4872fe56-centos7-x86_64-clang12/installed/asan/libcxx/include/c++/v1/thread:307:16 #3 0x7f463a7491af in yb::client::AsyncClientInitialiser::Start() /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/client/async_initializer.cc:77:25 #4 0x7f463fb6451f in yb::pggate::PgApiImpl::PgApiImpl(yb::pggate::PgApiContext, PgTypeEntity const*, int, PgCallbacks) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:260:22 #5 0x7f463fb2d555 in yb::pggate::YBCInitPgGateEx(PgTypeEntity const*, int, PgCallbacks, yb::pggate::PgApiContext*) /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:115:17 #6 0x7f463fb2d9b0 in YBCInitPgGate /nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:123:3 #7 0x1437e54 in YBInitPostgresBackend /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/utils/misc/../../../../../../../src/postgres/src/backend/utils/misc/pg_yb_utils.c:497:3 #8 0x13f014c in InitPostgres /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/utils/init/../../../../../../../src/postgres/src/backend/utils/init/postinit.c:0:3 #9 0xfeaaa5 in PostgresMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/tcop/../../../../../../src/postgres/src/backend/tcop/postgres.c:4707:2 #10 0xe38f6c in BackendRun /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4470:2 #11 0xe37d75 in BackendStartup /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:4136:3 #12 0xe35b35 in ServerLoop /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1754:7 #13 0xe3213a in PostmasterMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/postmaster/../../../../../../src/postgres/src/backend/postmaster/postmaster.c:1417:11 #14 0xc22999 in PostgresServerProcessMain /nfusr/dev-server/dfelsing/code/yugabyte-db/src/postgres/src/backend/main/../../../../../../src/postgres/src/backend/main/main.c:234:3 #15 0xc230a1 in main ??:0:0 #16 0x7f463e535554 in __libc_start_main ??:0:0 SUMMARY: AddressSanitizer: heap-use-after-free (/nfusr/dev-server/dfelsing/code/yugabyte-db/build/asan-clang12-dynamic-ninja/lib/./libyb_client.so+0x11b2765) Shadow bytes around the buggy address: 0x0c047fffc9b0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffc9c0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffc9d0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffc9e0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffc9f0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd =>0x0c047fffca00: fa fa[fd]fa fa fa fd fd fa fa fd fd fa fa fd fa 0x0c047fffca10: fa fa fd fa fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffca20: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffca30: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffca40: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd 0x0c047fffca50: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==18764==ABORTING