I have many projects that will never have business objects.  This adds quite a bit of time when the Business Object Mapper is launched from within Visual Studio.

The key is not to use a project reference....that is why it breaks when you combine them into multiple solutions.  It is somewhat of a pros/cons type of approach.  What we do is have all of our application specific assemblies compile to the same output location, for example:

C:\Development\MyApplication Build

Then each of the projects reference the assembly in that build folder versus a project reference.  This will actually speed up your IDE as well...by a lot in some cases.  However, the downside is that when you right-click a class and choose "Go To Definition" it will not be able to if the class is in a different project.  But that is the only downside...which we decided wasn't too bad since generally you are doing this more often for classes within the same assembly.

Next, you will need to be sure and set your build order on your solution so that if you pull everything back down from source control...clear out the build folder, etc. it will build in the proper order so that when the dependent assemblies are built before they are needed as a reference, etc.