Infinite recursion clang11 AST crash

[jlisee]

While parsing a source file (will work to find the one) the libclang-cpp.so.11 gets stuck in a infinite loop between getTypeInfo and getTypeInfoImpl. There are several infinite recursion bugs fixed in clang since version 11, hopefully the debugging tool can pin point it so it can be removed.

Full stack trace:

#149284 0x00007ffff581a0d3 in clang::ASTContext::getTypeInfo(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149285 0x00007ffff5819301 in clang::ASTContext::getTypeInfoImpl(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149286 0x00007ffff581a0d3 in clang::ASTContext::getTypeInfo(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149287 0x00007ffff5819f93 in clang::ASTContext::getTypeInfoInChars(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149288 0x00007ffff5b2a05d in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149289 0x00007ffff5b25e10 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149290 0x00007ffff5b1f643 in clang::ASTContext::getASTRecordLayout(clang::RecordDecl const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149291 0x00007ffff5b1e7f4 in clang::ASTContext::getASTRecordLayout(clang::RecordDecl const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149292 0x00007ffff58193ef in clang::ASTContext::getTypeInfoImpl(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149293 0x00007ffff581a0d3 in clang::ASTContext::getTypeInfo(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149294 0x00007ffff5819f93 in clang::ASTContext::getTypeInfoInChars(clang::Type const*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149295 0x00007ffff59fce26 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149296 0x00007ffff5a12e40 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149297 0x00007ffff59ea47a in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149298 0x00007ffff5a13795 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149299 0x00007ffff59ea47a in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149300 0x00007ffff59eca41 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149301 0x00007ffff59ee184 in clang::Expr::isCXX11ConstantExpr(clang::ASTContext const&, clang::APValue*, clang::SourceLocation*) const () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149302 0x00007ffff59ed083 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149303 0x00007ffff5e30d0a in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149304 0x00007ffff5e17946 in clang::Sema::CheckImplicitConversions(clang::Expr*, clang::SourceLocation) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149305 0x00007ffff5e18244 in clang::Sema::CheckCompletedExpr(clang::Expr*, clang::SourceLocation, bool) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149306 0x00007ffff60d399b in clang::Sema::ActOnFinishFullExpr(clang::Expr*, clang::SourceLocation, bool, bool) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149307 0x00007ffff5f99876 in clang::Sema::BuildStaticAssertDeclaration(clang::SourceLocation, clang::Expr*, clang::StringLiteral*, clang::SourceLocation, bool) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149308 0x00007ffff5788528 in clang::Parser::ParseStaticAssertDeclaration(clang::SourceLocation&) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149309 0x00007ffff579067b in clang::Parser::ParseCXXClassMemberDeclaration(clang::AccessSpecifier, clang::ParsedAttributes&, clang::Parser::ParsedTemplateInfo const&, clang::ParsingDeclRAIIObject*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149310 0x00007ffff5791bc1 in clang::Parser::ParseCXXClassMemberDeclarationWithPragmas(clang::AccessSpecifier&, clang::Parser::ParsedAttributesWithRange&, clang::TypeSpecifierType, clang::Decl*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149311 0x00007ffff578ceab in clang::Parser::ParseCXXMemberSpecification(clang::SourceLocation, clang::SourceLocation, clang::Parser::ParsedAttributesWithRange&, unsigned int, clang::Decl*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
---Type <return> to continue, or q <return> to quit---
#149312 0x00007ffff578ac6d in clang::Parser::ParseClassSpecifier(clang::tok::TokenKind, clang::SourceLocation, clang::DeclSpec&, clang::Parser::ParsedTemplateInfo const&, clang::AccessSpecifier, bool, clang::Parser::DeclSpecContext, clang::Parser::ParsedAttributesWithRange&) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149313 0x00007ffff576fad0 in clang::Parser::ParseDeclarationSpecifiers(clang::DeclSpec&, clang::Parser::ParsedTemplateInfo const&, clang::AccessSpecifier, clang::Parser::DeclSpecContext, clang::Parser::LateParsedAttrList*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149314 0x00007ffff580176a in clang::Parser::ParseDeclOrFunctionDefInternal(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec&, clang::AccessSpecifier) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149315 0x00007ffff580144c in clang::Parser::ParseDeclarationOrFunctionDefinition(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*, clang::AccessSpecifier) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149316 0x00007ffff58006d6 in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149317 0x00007ffff5785a8a in clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149318 0x00007ffff5785697 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149319 0x00007ffff576f22c in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::Parser::ParsedAttributesWithRange&, clang::SourceLocation*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149320 0x00007ffff57ffe1f in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149321 0x00007ffff57fec2b in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149322 0x00007ffff57612ad in clang::ParseAST(clang::Sema&, bool, bool) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149323 0x00007ffff6d686d8 in clang::FrontendAction::Execute() () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149324 0x00007ffff6d1fb21 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149325 0x00007ffff6ed06fc in clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149326 0x00007ffff6ed0466 in clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) ()
   from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149327 0x00007ffff6ecf954 in clang::tooling::ToolInvocation::run() () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149328 0x00007ffff6ed1938 in clang::tooling::ClangTool::run(clang::tooling::ToolAction*) () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149329 0x00007ffff6eb848c in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libclang-cpp.so.11
#149330 0x00007ffff08d902f in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libLLVM-11.so.1
#149331 0x00007ffff08c16c7 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libLLVM-11.so.1
#149332 0x00007fffefc7e907 in __pthread_once_slow (once_control=0x4f9bea8, init_routine=0x7ffff7b068a0 <__once_proxy>) at pthread_once.c:116
#149333 0x00007ffff08c1660 in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libLLVM-11.so.1
#149334 0x00007ffff08d8dfd in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libLLVM-11.so.1
#149335 0x00007ffff08d88be in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libLLVM-11.so.1
#149336 0x00007ffff7b076df in ?? () from /var/lib/buildkite-agent/builds/ci-large-staging-agent-queue-i-0ab183a7e562f41cb-1/aurora-tech/av-buildkite-interactive/external/lsif-clang/libstdc++.so.6
#149337 0x00007fffefc766db in start_thread (arg=0x7fffa3ff6700) at pthread_create.c:463
#149338 0x00007fffef3e961f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[jlisee]

So I saw the utils/lsif-clang-driver.py tool from 9b6beab and ran it on my tree in an attempt to find the source of the segmentation faults. My results are 1331/34898 lsif-clang commands failed. 😭 but none of them actually segfaulted. Those are all compilation errors.

I am going to make one more modification to ensure I have captured any possible segfaults.

[jlisee]

I was able to succesfully index the tree with my clang 13 branch here https://github.com/jlisee/lsif-clang/tree/jlisee/llvm-13.0.1-upgrade-v2

Changes were basically zero besides updating the upstream code to version 13 and changing the build system to work with it.

EDIT - The modification I made to the drive did not seem to fix anything.

[varungandhi-src]

EDIT - The modification I made to the drive did not seem to fix anything.

Could you clarify what you mean by this?

[jlisee]

I updated the driver to make sure it caught any segfaults that might of been happening and there were in fact none. This indicates the problem was possibly related to issues with mulit-threaded processing.

Anyways I think updating to a newer version of clang and clangd as base is the way to go because it has gotten more stable over time so will provide overall more stability for the project.

[varungandhi-src]

Leaving this issue open in case you or someone else finds a reproduction. I've created #89 for the potential upgrade.