37 replies to this topic

[DDH69]

The shapes stuff is a part I never use myself, so your input on how it's supposed to work is definitely helpful.  Given that the DirectOutputShapes.* files have always been installed in the main folder, I'd say we should keep the install as-is and fix the part of the code that searches for them.  If it's just the DirectOutputShapes.* files, it should be fairly easy to find all the places in the code where they're loaded.  I'll definitely add some logging while I'm at it - my guess is that DOF considers those files "optional", so it doesn't consider it worthy of a warning if they're not found.  But that's the wrong idea since it's not helpful for troubleshooting if you wanted them to be found.  My attitude on logging in DOF is the more the merrier, since it's so hard to debug something with no UI running in the background.

 

Agreed.  Happy to retest, just let me know.

 

BTW - new version going well with DOFLinx other than this little issue.

[mjr]

DDH69 - actually, one thing that would be helpful to know from your log.  Could you search for this:

 

Install folder lookup: install folder is ...

 

... and tell me what that whole line says?

 

It looks like it actually *is* looking in the root DirectOutput folder, which is the folder identified in the line above.  That *should* be the common install folder, the parent of the x86/x64 folders where the DLLs are located, but maybe it's getting that wrong for your setup.  If so that might the real root of the problem, so a different thing to investigate.  But if it has it right then there's something I'm missing in the file lookup for the shapes files.

[DDH69]

2024.09.06 08:49:37.830 Loading global configuration from C:\DirectOutput\config\GlobalConfig_b2sserver.xml

2024.09.06 08:49:37.846 Install folder lookup: install folder is C:\DirectOutput; this is the ASSEMBLY folder -> original flat install configuration

2024.09.06 08:49:37.846 Global config successfully loaded from C:\DirectOutput\config\GlobalConfig_b2sserver.xml

 

So this is DOFLinx opening DirectOutput from the \DOFLinx folder. You may not have anticipated that.  The installation I have is both x32 and x64 but using the x64.  A lot of people have DOFLinx in the same folder as DiectOutput.dll but with the new installation paths you have that will get really messy.  I've always run from a separate folder which worked, until now.  Having said that, even if I place DOFLinx in the same folder as DirectOutput.dll it still requires the shape files in the config folder.

[mjr]

We definitely need to come up with a convention here that everyone can follow to get deterministic results.  No heuristics if we can avoid it - let's come up with one simple rule that everyone can follow, like "DLLs go on x64/x86, everything else goes in the main install folder".

 

So, what's the relationship between DOFLinx and DirectOutput.dll?   Given that .NET is reporting the Assembly folder as DOFLinx's location, I take it that DOFLinx is the DLL that's actually running in this case?  But if that's true, then I don't understand this part:

 

> Having said that, even if I place DOFLinx in the same folder as DirectOutput.dll it still requires the shape files in the config folder.

 

It seems like if what I just said about DOFLinx being the active assembly is true, then moving it to ./x64 would change .NET's report of the Assembly location to ./x64 as well, and DOF would start finding the shapes files in the right place.

 

I must be making some wrong assumption about how DOFLinx works.  Can you fill me on why we're getting the Assembly folder reported this way in both cases?

I'm also now very unclear on why it's not looking for the shapes file in C:\DirectOutput in ANY of these scenarios.  As long DOF believes that C:\DirectOutput is the install folder, *regardless of how it came to that conclusion*, it should be looking for the shapes file there.  It does sound like we need some logging to see what's going on in the file search, or I just need to puzzle out what the code thinks it's doing more thoroughly.

Edited by mjr, 06 September 2024 - 04:46 AM.

[DDH69]

In DOFlinx's early days (nearly 10 year now!) it was just one EXE placed in the \DirectOutput folder to co-locate with the various DLLs.  Over time the only required DLL is now DirectOutput.DLL

 

With the extra parts of DOFlinx, its own sub folder structure, and the case that it is installed on many MAME cabinets without pinball that use only a small part of the DirectOutput Framework, the preferred method is now to install in \DOFLinx and leave \DirectOutput alone.  This unblurs the lines between the two installations a lot (same as your intension in splitting x86 and x64).  The only item that is copied to \DOFLinx is DirectOutput.dll . This is necessary as it cannot name that assembly in a DOFLinx.exe.config file as its not strongly named.  This has been a good layout and working well.

 

The change is now the new installation of \DirectOutput\x86 & \x64 .  This works with the \DOFLinx location and copy of DirectOutput.dll as described above just fine, except for the issue with the shape files.

 

So DOFLinx.EXE is the main assembly using DirectOutput.dll to open output devices directly, except for addressable LEDs where DOFLinx opens a cabinet pointing to the \DirectOutput\config\GlobalConfig_b2sServer.xml

 

I can not explain why placing DOFLinx in \DirectOutput\x64 to test this does not find the shape files in \DirectOutput . 

[mjr]

> I can not explain why placing DOFLinx in \DirectOutput\x64 to test this does not find the shape files in \DirectOutput . 

 

Mystery solved - it turns out it's a bug in a rather far-flung bit of code related to the shapes file lookup, but only indirectly, so I was looking in the wrong place initially.

 

I think with a fix for the bug, it will work perfectly well to keep the DOFLinx install configuration exactly as you have it.  Your first log file report where it identified the install folder as C:\DirectOutput SHOULD have worked, because the key to the whole thing is correctly identifying the install folder.  I tried to set it up from the start so that the DLLs would work under the new subfolder divisions OR the original "flat" configuration, by automatically sensing which configuration they were running in when they executed.  So when DOFLinx is the "assembly" running in the root install folder, DOF should sense the "flat" configuration is in use and look for config and asset files there.  That's what was confusing me - your log seemed to show it was finding the correct folder, but despite that, it wasn't finding the asset file.  I should have realized from that alone that the bug was just in the asset file lookup, but once I added some logging it became clear what was going on.  I made some updates to the asset file search code (to accommodate the new install folder layout) and got a detail wrong in there, which made the file search for the Shapes file fail.

 

I've uploaded a test build with the fix - this also adds logging for the Shapes file lookup, so in case this doesn't solve it for you, it should at least give us more information.  But I think I already have a good handle on what was going wrong now, so I'm hoping it's just fixed now.

 

http://mjrnet.org/pi...se-20240905.msi

 

http://mjrnet.org/pi...se-20240905.msi

[DDH69]

Thanks mjr

 

Just tested and same end result.  More logging, and the install folder now seen as \DOFLinx

 

I've attached a full log with shapes in config and working plus just in \DirectOutput and not working.  I can see that the not working test is seeing \DOFLinx as the install folder. Clearly previous versions have seen \DirectOutput as the folder to look for the shape files regardless of where the dll is located or run from.  How do we get a neat way to recreate the use of that location with your new logic?  Should the path for the global config file be determining the install folder not where its being run from?

 

 

Attached Files

•  DirectOutput_Logs_Shapes.zip   24.23KB   1 downloads

[mjr]

Oh, I see, you're explicitly telling it to load the config file from a separate folder.

 

What mechanism exactly are you using to do that?  I want to make sure I can see how it traces through the code.

 

From my perspective, i think the cleanest solution would be to eliminate your extra copy of the DLL.  I understand that the issue is that you can't tell the Windows loader where to look for a hard-linked DLL.  The solution to that is to not hard-link it, but load it dynamically from your own code, via LoadLibrary().  That's a bit more work, but it gives you control over the source location of the DLL, so you can read it out of your own config file, for example, or read the location of the DOF COM object from the registry.

 

If DOFLinx is written in C#, you could also consider changing from linking to DirectOutput.DLL and instead use the DOF COM object, if that has the API calls you need.  C# should make it pretty easy to instantiate the DOF COM object as a client, and that would let COM take care of finding the non-co-located DLL for you.  If you're using C/C++, switching to LoadLibrary/GetProcAddress would probably be easier; you could do that with a simple glue layer that would leave the rest of the code unchanged. You could probably do something similar in C#, actually, so maybe that would be the easiest way in either case.

 

I'm not saying it's "cleanest" just because it's less work on the DOF side - it's more that I'd prefer not to introduce new rules about where files are found on the DOF side, because that increases troubleshooting workload for users.  I think it also benefits DOFLinx to do it that way, because as it, that extra copy of the DLL is not ideal, right?  It's an extra install step and a maintenance hassle for users to keep it in sync with the main DOF install, and something to go wrong troubleshooting-wise.

 

 

Are you up for that?

Edited by mjr, 06 September 2024 - 07:26 PM.

[DDH69]

When I open for using addressable LEDs (the relevant item for the shape file issue)

 

Pinball.Setup(GlobalConfigFile_Full_Path,"",TheRom)

Pinball.Init()

 

for all output devices I open them directly, Pinscape being one of the simpler ones (for context)

DOFDevice=DirectOutput.Cab.Out.PS.Pinscape(UnitNumber)

 

Regardless of DOFLinx copying or loading the DLL from \32 or \x64 the file location logic issue needs resolution still.  The base folder will remain as \DOFLinx in both circumstances.

 

Not copying the DLL is another issue with solution now made easier by the \x32 \x64 pathing logic you are implementing.  FYI, the code is all VB DotNet.  I have logic for run time loading of other DLLs, doing this for DirectOutput has never been advantageous until now whereby chasing versions and matching bit types can be eliminated with DOFLinx automatically selecting the correct DLL to match the running bit size of the application.  Re-engineering this is substantive work that will be tackled later, not a solution to the current issue.

[mjr]

If you load the DLL from the native folder, I think DOF will find its config files in the right place.  DOF isn't looking in the EXE folder - it looks in its own self-relative DLL folder.  The reason it thinks the DOF folder is C:\DOFLinx in your setup is that the running DirectOutput.dll is the one in your C:\DOFLinx folder.  The demo program below will verify this for you if you run it and inspect the DirectOutput.log file.

 

Take a look here for a mechanism that will let you load the native DLL pretty effortlessly - this shouldn't require any changes to your existing code, just a small addition.

 

https://github.com/m...lientImportDemo

 

I can see what you're saying about the version dependency issue.  As a rule I like to just distribute private copies of all required DLLs with my programs, to avoid the inevitable version conflicts with using shared copies.  But DOF seems different to me because of the config file dependencies.  If the demo above makes it as easy to use the native DOF DLLs as I'm claiming, are the version conflict concerns alone important enough to you to complicate things on the DOF side?  My big concern here is the impact on users from having to add new rules about how DOF finds its files.

Edited by mjr, 07 September 2024 - 12:06 AM.

[DDH69]

mjr, FYI, the current version you have is also missing the latest Dude's Cab work.

[mjr]

> mjr, FYI, the current version you have is also missing the latest Dude's Cab work.

 

It includes the latest they've sent me.  If you have something newer, feel free to open a pull request.

 

("Also"?)

[DDH69]

> mjr, FYI, the current version you have is also missing the latest Dude's Cab work.

 

It includes the latest they've sent me.  If you have something newer, feel free to open a pull request.

 

("Also"?)

 

yes, "also" not fully backward compatible as per the previous conversation.

[mjr]

> yes, "also" not fully backward compatible as per the previous conversation.

 

Ball's in your court on that one, right?  You can add that code snippet I provided to load the shared DLL - no re-engineering required, just some copy-and-paste.

[DDH69]

> yes, "also" not fully backward compatible as per the previous conversation.

 

Ball's in your court on that one, right?  You can add that code snippet I provided to load the shared DLL - no re-engineering required, just some copy-and-paste.

 

I apologise if you were left with that impression.  No its not that simple, yes there is re-engineering.

 

While I agree that there are alternative ways to handle the use of a DLL, the change is not as simple as you articulate.  For example the code you provided is similar to what I do for other DLLs already, but, for example, it does not work when applied in an x64 environment with on older installation of DOF, it returns paths to .Net Framework items, not DOF.  There are many scenarios to consider and address so arbitrary change is not a wise choice., DOFLinx has become heavily linked to DirectOutput over the last 10 years, a simple loading of the assembly is about 5% of the overall effort to change to a new way of working hence why backward compatibility is essential.

 

In short, there are many nuances to address.  I will of course, as always, seek to go forward, but ultimately as the person making changes to DOF, I feel you have the responsibility to ensure that the new changes will not leave existing users strandard, forced to upgrade or with a work around.  I will be making changes, but even when I do, there are many thousands of people who will break their machines by innocently following the path of upgrading DOF (due to Pinscape firmware or otherwise) then not knowing why other seemingly unrelated items stop working.  I hope we can continue to work together to make advancements that consider where people are now as well as where we want to take them going forward.

[mjr]

> for example, it does not work when applied in an x64 environment with on older installation

> of DOF, it returns paths to .Net Framework items, not DOF

 

Let me ask you this: do you have a way to resolve the path to the shared DOF DLL that you want to load?

[DDH69]

>> Let me ask you this: do you have a way to resolve the path to the shared DOF DLL that you want to load?

Yes.

 

I apologise if my Aussie accent is confusing things, please allow me to attempt to clarify.

 

Firstly, please do not be concerned about how I will deal with loading DOF in the new x32 x64 setup you are creating, I will address that later in a future DOFLinx release.

 

The issues I have are about the DOF release you have in beta, that as previous, will likely become the definitive DOF installation used by the community.  This beta, as it stands, will cause people’s systems to stop working in a couple of areas if installed.  I am sure no one wants that.

The issues I am aware of at present are:

1. The path checking for files is not checking the base installation directory when run from outside of that structure.  Like it or not this is a common installation layout and a new DOF release should not be made that will knowingly not be backward compatible and break addressable LEDs for people.
2. The latest updates for Dude’s Cab are not included.  Again, if people load the beta as you have it at present it will break their installation if they have a Dude’s Cab.

 

I am continuing to work with the new beta installations and have struck a new issue that I could use some advice on.  The scenario is:

1. Brand new machine, Win 11 64 bit
2. Installed both x86 and x64 msi installers (both so tat I can test DOFLinx x32 and x64)
3. The COM is registered for x86 but not x64, meaning GetTypeFromCLSID will work for x32 but not x64
4. Have tried uninstall / reinstall and repair
5. Have tried uninstalling both and then only reinstalling x64
6. Have tried running RegisterDirectOutputComObject.exe manually from the x64 folder, it returns an error 100.  I have no idea what this means.

[mjr]

>> Let me ask you this: do you have a way to resolve the path to the shared DOF DLL that you want to load?

> Yes.

 

Okay, in that case, please indulge me.  Could you open the demo project I opened earlier, go to DofDllImportVB / Module.vb, and scroll down to the line that looks like this:

 

Dim dofBasePath As String = "d:\per\temp\dof222"

 

Now, if you replace that string for dofBasePath with the path that you've already resolved as the DOF load path, can you tell me why that won't work for you to load the DOF DLL from the shared folder?

 

 

 

> The COM is registered for x86 but not x64, meaning GetTypeFromCLSID will work for x32 but not x64

 

Could you look at the registry and check these two keys:

 

HKEY_CLASSES_ROOT\CLSID\{A23BFDBC-9A8A-46C0-8672-60F23D54FFB6}

 

HKEY_CLASSES_ROOT\WOW6432Node\CLSID\{A23BFDBC-9A8A-46C0-8672-60F23D54FFB6}

 

Tell me what you see there.

You can also try running the MSI in verbose log mode:

 

DirectOutput-mjr-x64-release-xxxx.msi /L*V setup.log

 

Then check the setup log and look for the registry string, see if it's reporting any errors.  The registry updates are handled by MS Setup, so I'd think Setup would always display an error in the UI if it encounters any problem, but maybe there's some condition where it fails silently and just logs it.

Edited by mjr, 10 September 2024 - 05:35 PM.