32 replies to this topic

[bgood]

Keep adjusting and move your head less, Just wait till you get your cyan and red 3d glasses and run FP in 3D. Now with the latest VPX,  10.7.3  it's also amazing with Kinect in 3D. Glasses are available on Amazon for cheap. I'm using the Xbox Kinect so maybe there is a difference. I don't know but I do know that the Kinect made one big difference in the depth of the table perception even without being in 3D. I will not do VP and FP without my Kinect. OBTW your video card must be a 1050 or better to work properly. Hope this encouragement helps.

[pdev]

Hi all,

 

thank you for all the info in this thread, it is very informative. I'll share my experience in regard to the VR part.

I am somewhat a veteran of VR, having programmed games and tools (mostly archviz stuff) since the Oculus DK1 and when the first pinball games were playable in vr i was quite stunned away at the realism that the depth give!

however at the time I had the Oculus CV1 which was tethered via usb/hdmi so latency was minimal and, for me, this is the crucial part. I have tested various tables and the lacking part was the resolution of the cv1 (1080x1200 resolution per eye at 90Hz)

but everything else was just stunning. Fast forward to some years later and I tried playing using the Quest 2 both via WiFi and tethered via usb and .. alas, the latency ruined the magic for me. Even when tethered via usb the quest 2 has way too much latency for a pinball game since it compress and pushes the graphic via usb so even with recent tech, at least for me, the peak of playing pinball in vr was using the Oculus CV1.

of course there are standalone pinball game for the Quest 2 which are fun but the graphics is not on par with the pc ones so ... we'll see with the quest 3 if this gets better. also having the headset on and looking down really stress the neck after a while.

Nowadays I'm gathering all the parts for a vpin, meanwhile I'm playing with proper buttons attached to a wood controller which I made and for the screen I just rotate vertically my monitor which has a 1ms latency and 165Hz refresh rate and .. oh boy, seeing the fluidity of the ball rolling on the table is just incredible, so much that I just can't go back to less Hz without having the feeling of playing a sub-par videogame. But I really miss the 3d / depth effect that's why I have just ordered on ebay a Kinect to play around with the 3d effect,

having had some similar experience with this some years ago (search Johnny Lee wiimote on youtube) I know if tuned very well it can definitely add a 3d look to the whole simulation, but after reading your comments I'm a bit concerned about the whole "one eye" effect, since by playing with head tracking I didn't notice it lose depth when using both eyes. Anyway since from time to time a project popped up that was using the kinect (3d scan etc.. I'm also into 3d modeling) this time I took the plunge and ordered one from ebay

I really hope I can add a "somewhat" 3d effect to my setup since I almost finished gathering pieces and the next step will be building the whole thing.

[pdev]

Hi pals,

 

I'm writing a custom program for the head tracking and so far I managed to feed fake data to vpx (waiting for the kinect to arrive!) I'm tuning the values right now. why am I doing this? I just want to skip the BAM part altogether since the data is just written to a named shared memory anyway,

I need help, could someone give me some range for the values that gets read in bam-tracker when using the kinect or simplecam?

thank you

Edited by pdev, 20 October 2023 - 09:37 PM.

[pdev]

Nevermind, after many hours of frustrating experience with BAM (I just couldn't get it to load the head tracking plugin) I gave up BAM and started looking in the vpx source code to see how it communicated with BAM.

After some digging i found that it is using a named shared memory (found the relative code in the file BAM_Tracker.h inside "third-party/include/" folder, the relevant code is this:

    /// <summary>
    /// The shared memory / memory-mapped file name
    /// </summary>
    static const char *SharedMemoryFileName = "BAM-Tracker-Shared-Memory";

    /// <summary>
    /// Single captured data about player position.
    /// </summary>
    struct TPlayerData {
        double StartPosition[4]; // x,y,z [mm] + timestamp [ms]
        double EndPosition[4];   // x,y,z [mm] + timestamp [ms]
        double EyeVec[3]; // [normalized vector]
        int FrameCounter;
    };

so, basically I rolled my own head tracking solution with ctypes in python, then I used opencv to track my head and feed the data. all good.

here are some videos of the final result. in one of the video you can see what happens when the tracker stop working, I since figured out how to eliminate this problem (it was a webcam focus issue)

 

 

https://youtube.com/shorts/Sm7krZm-exo

 

 

https://youtube.com/shorts/PXvKNnezIT4

 

 

however there are a few considerations that I made:

 

- For my current setup (just a vertical monitor and some buttons) it is not worth it since my head is mostly still

- For achieving a good effect a BIG monitor is needed (with a 27" the effect is already good but I think 32" would just nail it)

- It really need to be installed on a cabinet and you need to look mostly down

- A better solution than a regular webcam is really needed (I managed to push the limit of my Logitech C615 by using compressed frame, b&w etc but ..) probably the Kinect would be just better since it only feeds the tracking data via usb and not a whole raster image (If I am correct, still waiting for my kinect to arrive)

- If you just keep using a webcam then install the relative software for your cam, in my case I used the logitech software to fine-tune the focus and managed to almost eliminate deadzones or tracking lost

aske me anything about this

if you're interested in the python (using ctypes) code, here's the relevant bit to set up the named shared memory in case you want to roll our own solution:

# Define the TPlayerData struct to match the C++ struct layout
class TPlayerData(ctypes.Structure):
    _fields_ = [
        ("StartPosition", ctypes.c_double * 4),
        ("EndPosition", ctypes.c_double * 4),
        ("EyeVec", ctypes.c_double * 3),
        ("FrameCounter", ctypes.c_int)
    ]

# Define the shared memory file name and size
SharedMemoryFileName = "BAM-Tracker-Shared-Memory"
SharedMemorySize = ctypes.sizeof(TPlayerData)

# Open the existing shared memory segment 
shared_memory = mmap.mmap(-1, SharedMemorySize, SharedMemoryFileName)

then it is just a matter of writing the position, something like this:
initial_data = TPlayerData()
initial_data.StartPosition = (0.0, 0.1, 0.4, 1.10) # <- Edit this value according to your setup
initial_data.EndPosition = (0.0, 0.1, 0.4, 1.10) # <- Edit this value according to your setup
initial_data.EyeVec = (1.0, 1.0, 20.0) # <- Edit this value according to your setup
initial_data.FrameCounter = 1
shared_memory.seek(0)
shared_memory.write(initial_data)
# BAM Integration

then on each frame alter the values the way you want and increase the FrameCounter before writing back the changes

I'll probably write on my blog about this, clean up the code a bit and put it on github when I have the time

long story short, I'll get back to polish this when I finally build my vpin

Edited by pdev, 21 October 2023 - 09:30 PM.

[mshaker]

Hi All; 

 

I've been using BAM & the Kinect 2 Head Tracking with both VPX and Future Pinball for several months. 

 

My set-up is less than optimal. I could not place the Kinect at the recommended position at the bottom of the backglass because of the style of my cabinet. Also, I have a control panel that pushes the screen back about 6in (15cm) and I'm tall. Because of this, I had to place the Kinect not just on top of my Backbox, but on a small tower, so that it could see both me and the necessary amount of the playfield. 

 

So...It's laggy. It's jerky. It sometime drops out to normal 2D play. It goes nuts when someone else walks by. It doesn't track turning your head... 

 

Despite all that: This is what I want for immersion. This is the future. I want to be in the same room with my friends, not enclosed in a VR headset.

For me: This 3D illusion is much more immersive that DOF or SSF.

 

I'm hoping that the experts here can provide me with some info: 

 

I've only used the Kinect 2: How does the experience with other hardware (PS3 Eye, Web Cam...) compare? 

 

Can the lag be reduced with a more powerful CPU or GPU?
(Or is the lag "built-in" to the Kinect?) 

 

Is there anyway to improve it's "lock" on the player's image, so it doesn't go nuts when someone else walks by?

 

Is there anyway to reduce the amount of the playfield that must be in the Kinects field of view (40% for top of backbox mount)?

(So it could put more of the player in it's field of view for more accurate tracking.) 
 

Are there any improvements incoming, such as tracking the direction the player's head is turned or supporting dual webcams (for stereoscopic tracking)?
 

Thanks for your help!

 

   - Mark

[sliderem]

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

[TerryRed]

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

Edited by TerryRed, 16 November 2023 - 01:48 AM.

[sliderem]

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

Super Thanks! I will give it a try. I still have 10.7 and I did not upgraded yet because is stable and I did not want to mess with it. I hope the upgrade will be without impacts

 

PS: I played with the Future Pinball a little bit, and the BAM  looks impressive!

Edited by sliderem, 18 November 2023 - 12:56 PM.

[Cadorna]

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

In reality, it is not necessary to exclusively use the VPGL version to enjoy the capabilities of Kinect 2, it works in exactly the same way as the "standard" DX version. In fact the 3D Stereo SBS functionality only works (not very well, but it works) in the DX version because it has not yet been implemented in the glide version.

[sliderem]

 

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

In reality, it is not necessary to exclusively use the VPGL version to enjoy the capabilities of Kinect 2, it works in exactly the same way as the "standard" DX version. In fact the 3D Stereo SBS functionality only works (not very well, but it works) in the DX version because it has not yet been implemented in the glide version.

 

I abandoned for a while to make this work in BPX 10.8, I do not think is stable enough. 

I have a less a less ambitious objective, to have this working in Future pinball perfectly.

In fact it works very well in FP, but I did not had any future pinball table, and I am struggling now to have everything good in FP : DOF, SSF,DND and back screen for some table .

[Cadorna]

 

 

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

In reality, it is not necessary to exclusively use the VPGL version to enjoy the capabilities of Kinect 2, it works in exactly the same way as the "standard" DX version. In fact the 3D Stereo SBS functionality only works (not very well, but it works) in the DX version because it has not yet been implemented in the glide version.

 

I abandoned for a while to make this work in BPX 10.8, I do not think is stable enough. 

I have a less a less ambitious objective, to have this working in Future pinball perfectly.

In fact it works very well in FP, but I did not had any future pinball table, and I am struggling now to have everything good in FP : DOF, SSF,DND and back screen for some table .

 

The head tracking capabilities using Kinect 2 in VPX have the same stability as FP because it essentially uses the same files. The 3D effect in FP is much better, I could say a 9.5/10 but it is not directly related to Kinect technology but rather to the way in which the games have been developed in my opinion.

[sliderem]

 

 

 

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

In reality, it is not necessary to exclusively use the VPGL version to enjoy the capabilities of Kinect 2, it works in exactly the same way as the "standard" DX version. In fact the 3D Stereo SBS functionality only works (not very well, but it works) in the DX version because it has not yet been implemented in the glide version.

 

I abandoned for a while to make this work in BPX 10.8, I do not think is stable enough. 

I have a less a less ambitious objective, to have this working in Future pinball perfectly.

In fact it works very well in FP, but I did not had any future pinball table, and I am struggling now to have everything good in FP : DOF, SSF,DND and back screen for some table .

 

The head tracking capabilities using Kinect 2 in VPX have the same stability as FP because it essentially uses the same files. The 3D effect in FP is much better, I could say a 9.5/10 but it is not directly related to Kinect technology but rather to the way in which the games have been developed in my opinion.

 

#

I do not know why in my case it seems it sets the view point wrong in VPX and head tracking is not working.

 

Exactly like here :

 

https://www.reddit.c..._the_squashing/

[Cadorna]

 

 

 

 

 

Dear all,

 

Just found this topic, and from what I see it looks amazing. I have a cabinet with VPX 10.7, I tried with red/blue glasses but I want to tray also this BAM with the Kinect 2.0.

 

Is there a clear (and newer) guide on how o install BAM on VPX ?

I found this but is from 4 years ago, and I am afraid that is not anymore up to date.

 

 

Best regards,

 

The newer versions of VPX 10.8 beta have BAM headtracking built-in. You need to run the VPGL version included to get that feature. Keep in mind, this is still beta, and many things are "in progress"... as well as it doesn't have the same level of support and options that FP-BAM has for these features.

 

In reality, it is not necessary to exclusively use the VPGL version to enjoy the capabilities of Kinect 2, it works in exactly the same way as the "standard" DX version. In fact the 3D Stereo SBS functionality only works (not very well, but it works) in the DX version because it has not yet been implemented in the glide version.

 

I abandoned for a while to make this work in BPX 10.8, I do not think is stable enough. 

I have a less a less ambitious objective, to have this working in Future pinball perfectly.

In fact it works very well in FP, but I did not had any future pinball table, and I am struggling now to have everything good in FP : DOF, SSF,DND and back screen for some table .

 

The head tracking capabilities using Kinect 2 in VPX have the same stability as FP because it essentially uses the same files. The 3D effect in FP is much better, I could say a 9.5/10 but it is not directly related to Kinect technology but rather to the way in which the games have been developed in my opinion.

 

#

I do not know why in my case it seems it sets the view point wrong in VPX and head tracking is not working.

 

Exactly like here :

 

https://www.reddit.c..._the_squashing/

 

Follow these steps:

 

1) Did you copy the entire folder from, for example, C:\Future Pinball\BAM to C:\Visual Pinball\BAM?

2) After that, did you execute the BAM-Tracking.exe from the C:\Visual Pinball\BAM folder?

3) After that, did you execute the VPX 10.8 beta version with head tracking support?

4) After that, did you check to enable the head-tracking option in the video preferences in VPX 10.8?

5) After that, while the table is running, did you hit the Q key to pop up the options menu and enter into the Adjust head tracking option menu?

6) After that, did you resize your table according to your position?