Merging/Unification of windows and trunk builds

Subprojects LLDB
1

Hello,

Many of the changes in the windows branches are actually contained in this patch (r189107).
Did you have any changes on top of the windows branch?

Since I work on the same stuff currently, I better lay out what I am doing and what’s my current current status, so that there is no duplication of efforts. I hope we can work together on this!

I recently restarted the windows port from scratch using what’s already been done in the windows branch to first support MinGW and then MSVC with incremental changes, so that it can be merged step by step into trunk (much more easy to get it merged than a single huge patch).

Also, a MinGW build is useful since it could be more easily added to the buildbot and test suite.

And since MinGW and MSVC changes have many common parts, I figured it would be good to do both of them in the proper order (more easy to add MSVC on top of MinGW than the opposite – windows branch was only targeting MSVC as far as I know, so it wasn’t making difference between #ifdef _WIN32 and #ifdef _MSC_VER).

The first MinGW patch just got accepted, but I still got the MSVC patch that I need to clean up and get reviewed.

Hopefully this should result in a much smaller and cleaner patch for MSVC, since it was done together with MinGW changes in mind. Also it should avoid the merging (which might be difficult on top of latest changes and MinGW patch).

Note that the MinGW patch also includes some changes that were not part of the windows branch to provide a better windows support (esp. file case resolution, etc…).

I currently target MSVC12 since it is supposed to have better C++11 support, but going from MSVC12 to MSVC11 is only a few changes.

If everybody is OK to go this way, most of the windows branch will end up being merged.
If people are interested in helping, I could publish the branch so we could work on it together.

After that there might still be some changes in the windows branch that I didn’t do, so it would be good to evaluate what’s left (but probably not so much).

As soon as python support is added, the lldb executable and the test suite should work as well.

Now, I happen to be working on the lldbProcessWindows/lldbDynamicLibraryWindows plugins. Many features are working (stack trace, breakpoints, stepping, disassembly, threads, locals, etc…).
I currently use it in a MSVC DebugEngine plugin. It’s still early stage but it’s starting to work.

Let me know what you think!

Virgile

2

I currently target MSVC12 since it is supposed to have better C++11
support, but going from MSVC12 to MSVC11 is only a few changes.

If everybody is OK to go this way, most of the windows branch will end up
being merged.
If people are interested in helping, I could publish the branch so we
could work on it together.

After that there might still be some changes in the windows branch that I
didn't do, so it would be good to evaluate what's left (but probably not so
much).

I am OK with this, but better talk with Carlos to make sure you get all of
the fixes he has been piling on top of the Windows branch. I'll try to test
your patches with MSVC11 and report whatever problems are found.

Now, I happen to be working on the

lldbProcessWindows/lldbDynamicLibraryWindows plugins. Many features are
working (stack trace, breakpoints, stepping, disassembly, threads, locals,
etc...).
I currently use it in a MSVC DebugEngine plugin. It's still early stage
but it's starting to work.

Let me know what you think!

This sounds awesome. I'd love to give it a try, are you open-sourcing the
plugin?

Also are you re-using any of the work that was open-sourced by Embarcadero
for the port? I only gave it a quick glance, but it seemed to have a lot of
code that could be re-used.

3

Yes sure, I keep in touch with Carlos. He’s been very helpful and supportive.

MSVC11 changes are related to the lack of , and some template instantiations issues, so not so huge different but that might be enough so that it is better kept as a separate patch/branch until everybody migrate to MSVC12? (MSVC11 doesn’t support full C++11 which LLDB targets, so if we want to keep trunk clean from those issues it seems to be the only option).
Note that you can compile it in MSVC11 with toolset vc120.

For lldbProcessWindows, I will merge it to LLDB.
For the MSVC AD7 debugengine, not sure yet if I will open-source it or do a low-cost commercial product out of it.

Didn’t have a chance to check embarcadero code yet, I will do that as soon as I am finished with the windows patches. However, it seems they took a different approach (using windows implementation of POSIX functions).

Sincerely,
Virgile

4

Hello all,

Recently, there has been quite some activity towards more Windows support in LLDB. We have been working on this for a while and have created a patch based on the Windows branch and the changes Virgile has been committing to trunk.

The aim being for thepatch tosuccessfully build in Visual Studio 2012for those developers who want 'native' windows support.The November CTP version of the Visual Studio 2012 compiler has to be used due to the recent C++11 changes in trunk.

We've created an LLDB driver as well for Windows by removing the editline dependencyon Windows.This is just support for the lldb library itself, we have not added on-windows debugging. We primarily use Windows LLDB with the remote plugins.

The patch will be ready soon as we're doing a final cleanup and we'll submit it shortly.

Thanks,
Deepak

5

Nice to hear, Deepak,

Are you in a position to run the test suite on Windows? Have you considered breaking down the work into more focused patches?

I expect that LLDB will be branched in early November (around the time of the next llvm developer conference). That gives folks a chance to meet in person to resolve release blockers. So, it would be good to aim for a stable trunk on that time-frame,

  • Ashok
6

Hi,

We cannot run the test suite on Windows as we've not added native debugging on Windows. A lot of the tests compile a native executable which will not work in Windows. One idea to go about testing would be toadda remote target such as connecting to a remote debugserver.

As a first focused patch, we can consider the LLDB command-line driver. The current driver depends on libeditline, which is not available on Windows. The version we have developed contains a lot of #ifdefs, generally because editline has been integrated quite deeply into the driver.

Would it be acceptable to add #ifdefs around most of the libedit code to get it workingon windows, or would refactoring more of the library into IOChannel from Driver to allow for a windows-targeted IOChannelbemore ideal?
We pretty much have a version based on #ifdefs ready to go.Refactoring would be cleaner in the long run but take more time to develop.

Thanks,
Deepak

7

We cannot run the test suite on Windows as we've not added native debugging on Windows. A lot of the tests compile a native executable which will not work in Windows. One idea to go about testing would be to add a remote target such as connecting to a remote debugserver.

Are you able to apply your patches and test the build and run the suite on other platforms such as 64-bit Ubuntu? Even if the community helps with pre-commit testing, we'll need to team up to debug regressions. When I say 'other platforms', a minimum are the platforms that are tested on the buildbots. It's always nice to test manually on OS/X as well, but that is getting less valuable given the current failures. I assume that a FreeBSD buildbot and Linux i386 build stages are around the corner as well.

Separately, it would be good to work as a community to add remote testing to the suite. This is now more practical because of lldb-platform for two reasons. We can automate more of the platform discovery and test setup (i.e. uploading of a debug server and the application to debug). Secondly, lldb-platform provides a decent fraction of what we are missing in gdbserver. So, it would be practical to test on a wider range of platforms using gdbserver + lldb-platform + a register definition file. The same tests should run against debugserver with a slightly different harness.

As a first focused patch, we can consider the LLDB command-line driver. The current driver depends on libeditline, which is not available on Windows. The version we have developed contains a lot of #ifdefs, generally because editline has been integrated quite deeply into the driver.

Would it be acceptable to add #ifdefs around most of the libedit code to get it working on windows, or would refactoring more of the library into IOChannel from Driver to allow for a windows-targeted IOChannel be more ideal? We pretty much have a version based on #ifdefs ready to go. Refactoring would be cleaner in the long run but take more time to develop.

I'll let the code-owner for the command-line driver (Greg or Jim) make the call here. If you check-in the #ifdef edition now, would that be with the understanding that there will be a more maintainable solution later? Incidentally, did you consider using the FreeBSD licensed MinGWEditLine?
    http://mingweditline.sourceforge.net/?Description

Thanks, Deepak

Nice to hear, Deepak,

Are you in a position to run the test suite on Windows? Have you considered breaking down the work into more focused patches?

I expect that LLDB will be branched in early November (around the time of the next llvm developer conference). That gives folks a chance to meet in person to resolve release blockers. So, it would be good to aim for a stable trunk on that time-frame,

- Ashok

8

On that note, the FreeBSD buildbots (i386, amd64) have been running
for some time and are linked from http://lldb.llvm.org/build.html

Direct links:
http://llvm-amd64.freebsd.your.org:8010/builders/lldb-amd64-freebsd
http://llvm-amd64.freebsd.your.org:8010/builders/lldb-i386-freebsd

The i386 build is in rather poor shape on FreeBSD, but the amd64 build
had no unexpected failures until recently.

9

Hi,

We cannot run the test suite on Windows as we've not added native debugging on Windows. A lot of the tests compile a native executable which will not work in Windows. One idea to go about testing would be to add a remote target such as connecting to a remote debugserver.

This is what the "platform" stuff that we merged in will start to be able to do. There are going to need to be many modifications to the test suite to get it to run remotely. Most of them are making sure we involve the platform to install/upload/download files prior to running each test. Before running the remote tests a "lldb-platform" binary will need to be started on the remote system then in the test suite python code we would need to do the equivalent of:

(lldb) platform select remote-linux
(lldb) platform connect connect://remote.linux.box:1234

Now when we run the tests, we now know we are connected to a remote system and after each test suite build we would need to:
- compile in host
- install executable or shared libraries on remote host prior to running the test
- then run the program remotely (which should happen automagically if we are connected to a remote platform)
- get any files that are produced by the test (stdout redirected files, etc) back onto the host for the test suite to handle

A lot of the platform code has been started, and there is still work to be done to finish this off, but the work is _very_ worthwhile.

As a first focused patch, we can consider the LLDB command-line driver. The current driver depends on libeditline, which is not available on Windows. The version we have developed contains a lot of #ifdefs, generally because editline has been integrated quite deeply into the driver.

Would it be acceptable to add #ifdefs around most of the libedit code to get it working on windows, or would refactoring more of the library into IOChannel from Driver to allow for a windows-targeted IOChannel be more ideal?
We pretty much have a version based on #ifdefs ready to go. Refactoring would be cleaner in the long run but take more time to develop.

Abstracting it into a new class is the way to do. The main questions is if we try and build this into the LLDB shared library or into just the driver. Just the driver for now would suffice. Feel free to make a wrapper class for libedit as long as we don't lose any functionality that currently exists.

Greg

10

Quick update: I’ve just committed to trunk all the patches for MSVC12 (Visual Studio 2013 RC) build support.
It should now be possible to use cmake to generate a Visual Studio solution (.sln) and compile LLDB under both Visual Studio 2013 and MingW32. This include all LLDB libraries, not the lldb.exe driver though.
It’s probably easy to add Visual Studio 2012 support as well (just a few changes).

Unsupported:

  • lldb.exe driver (Deepak is working on it)
  • Python support

I will now focus my work on finishing and merging lldbProcessWindows, so that LLDB can debug win32 clang/gcc executables with LLDB. My current version already work for some simple executables so hopefully it shouldn’t be far away.
This should make LLDB usable in real scenarios/products for the Win32 LLVM/Clang toolchain.

Further along the road, it would also be great to support PDB (well, not even sure about legal aspect of it). It would probably be a priority only when LLVM/Clang supports the MSVC ABI (I heard there was some work in that direction), so that MSVC-compiled libraries can be used and debugged on windows.

Thanks again to all the people who helped for this integration (especially the authors of the original windows branch).

Virgile