DIY LED Sensor Light Gun

[twistedsymphony]

just connect to the Wii remote with bluetooth and save yourself lots of trouble!

IIRC there is some problem with using the remote as a whole. I believe it "cleans" the signal in a way that makes line of sight tracking more difficult.

not to mention if you want to retrofit it to an original arcade gun shell fitting the whole remote in there would be difficult.

though wireless guns would be fantastic

I did start playing around with it at one point but the project is on the back burner for the time being.

 

[bobbydilley]

Very old versions of OpenJVS did support the Wii remote (you can see it in my youtube video where I play maze of the kings), and I think I was just getting X and Y values for the 4 brightest LEDs it could see. I don't particuarly remember and smoothing or anything happening, but could be wrong. It did feel slightly strange though, and the lens seems to zoom in a lot making it so you have to stand far back to play :/

 

[bobbydilley]

I was bored at work today, and thinking about this thread so finally bit the bullet and bought the things required to attempt to remake a OMZ gun setup.

I've bought:

10 x IR LED Diodes
1 x Arduino Uno
1 x DFRobot Positional IR Camera

The IR camera that you can find here https://www.ebay.co.uk/itm/DFRobot-...e=STRK:MEBIDX:IT&_trksid=p2060353.m1438.l2649, although is now out of stock, is essentially the exact same sensor thats in a wii remote, but in a little module that you can connect strait to the i2c on an Arduino with no extra circuity.

My plan is to connect the 10 LEDs to the Arduino, and make a surround for my monitor with the LEDs in. I'll then flash them 1 by 1, and record the position of the points the camera can see. From those I can then calculate the points around the screen.

I'll keep the updates here as they go if thats okay with everyone on the thread? I will also most likely need assistance with the math for calculating the centre point.

If this works well, the total cost was about £35 for all the bits (less obviously a gun case) and so it would be a big advancement over the expensive OMZ price.

 

[twistedsymphony]

My plan is to connect the 10 LEDs to the Arduino, and make a surround for my monitor with the LEDs in. I'll then flash them 1 by 1, and record the position of the points the camera can see. From those I can then calculate the points around the screen.

solid plan. Looking forward to see where you go with this.

 

[winteriscoming]

@bobbydilley, please figure this one out for all of us!

I'm super interested in building out a solution for JVS gun games, but have too little free time, so if you can nail down the logic and hardware, that would be great!

As a stretch goal, consider implementing recoil support as a simple output that only activates when the gun is pointed at the screen (i.e. so offscreen reloading doesn't recoil).

 

[twistedsymphony]

As a stretch goal, consider implementing recoil support as a simple output that only activates when the gun is pointed at the screen (i.e. so offscreen reloading doesn't recoil).

you could do that today with the original OMZ setup.

since it outputs an analog signal for the gun position... when off-screen it still reports 0,0 for the position since it always has to report something, so since we can't know from that if the gun is actually off screen they include a separate digital output pin for "off-screen" (usually wired to one of the button inputs) you could run that to a relay to disable or enable the recoil depending on it's state.

if his solution is to be a drop in replacement for use on games like HOTD2 then it will have to implement that signal anyway

 

[bobbydilley]

I thought I would note the technical details of how I'm going to do the mapping here, so that people can pick into it and hopefully find some flaws that we can fix.

I'm going to imagine that this is the layout of the LEDs. (I called the last one 0 because 10 didn't fit well)

1 2 3 4
5====6
7 8 9 0

I'm going to attempt to find the 4 corners of the square (1, 4, 7 and 0)

If I can see them all then i'll just use those.
If I can see a sub-square, for example (2, 3, 8 and 9) then I will simply scale that square to fit.
If I can only see a triangle, so imagine (5, 1 and 2) I will calculate the vector to get from 1 to 5, and then the vector to get from 1 to 2. I will multiply the vector to get from 1 to 5 by 2 to get point 7, and multiply the vector to get from 1 to 2 by 3 to get point 4. I will then work out the vectors from 1 to 7, and 1 to 4 and apply both of these vectors to point 1 to get point 0.

Now I need to map the perspective of this quadrilateral defined by points (1, 4, 7 and 0) onto a unit square (a square with sides of length 1).

https://github.com/bobbydilley/OpenArcadeGun/blob/master/python_transform.py

The function above, takes in the 4 points of the quadrilateral and a target point (which will be 0.5, 0.5 as defined by the middle of the camera), and converts the target point onto the unit square. This should in theory get me the point on the screen between 0 to 1, which I can then scale to 255.

 

[winteriscoming]

I'm not sure the interpolation of the missing points in your example would account for a skewed angle. If you're short and pointing your gun at points 1 and 5 from a low point, from the perspective of the camera sensor, the distance between 5 and 7 would be greater than the distance between 1 and 5.

I don't know how big of a deal it will be to account for that kind of skew/perspective, but I believe you'd be able to interpolate missing points taking that into consideration.

Perhaps try it the way you're planning and see if it's accurate at most real-world angles and distances?

 

[twistedsymphony]

I would recommend ordering the leds in a loop. They're going to have to be chained that way anyway and that will also allow you to use off the shelf addressable leds too.

As for picking thr corners or a smaller square. I think that's a good strategy. But I agree with winteriscomming's point about skew.

 

[bobbydilley]

@twistedsymphony Ordering in a loop sounds like a good idea - I was just numbering it off the top of my head, but will likely start from 0 at the bottom left going clockwise!

@winteriscoming I totally agree with you there, however I can't see how it could be overcome with the 3 LEDs. Apologies for the bad drawing, but I think it illustrates your point.

Although we can clearly see that both black squares (representing the monitor) are viewed from a different positions, the lines inside the red square (the cameras view) are exactly the same. Without more information, such as other points, I don't believe its possible to determine which of the black squares we are looking at. If we did have more information such as another point on the bottom line, we would then have an entire quad which we could just enlarge to get the full black square, rather than using this triangle rule.

I'll see how well it works when I've got all the parts wired up, and then can think about the next steps of improving it; Sega/OMZ managed it so it must be possible!

 

[bobbydilley]

I was at Blackpool Pleasure Beach recently, and was looking at machines like Rambo and HOTD4, realized that they only have 5 LEDs on the top, and 5 LEDs on the bottom - and none up the sides as I originally thought they did.

I'm now slightly confused as to how they work - do they need to see an entire vertical slice of the screen? My idea was to look at triangles at the side, but they don't seem to be able to do that?

 

[twistedsymphony]

I was at Blackpool Pleasure Beach recently, and was looking at machines like Rambo and HOTD4, realized that they only have 5 LEDs on the top, and 5 LEDs on the bottom - and none up the sides as I originally thought they did.

only 1 game uses 12, and that's Jurassic Park the Lost World (The very first game to use this system). every game after that has only had 10.

the Gun Sense board actually has a switch to go between 12LED mode and 10LED mode.

If you're interested in numbering them the same way they go like this:

5 - 6 - 7 - 8 - 9

4 - 3 - 2 - 1 - 10

or for 12:

6 - 7 - 8 - 9 - 10
5               11
4 - 3 - 2 - 1 - 12

 

[bobbydilley]

Surely if you can only see the top row, there is no way to compute the vertical perspective as they're all on one axis?

Maybe the gun camera has a super wide field of view for the vertical plane so can always see LEDs at the top and bottom, but a rubbish field of view for the horizontal one, which is why they've got so many LEDs going across?

 

[bobbydilley]

So I've had a go with mixed results.

1 LED per reference point doesn't look like enough - It's not bright enough to be properly detected.
I've currently only tried 4, because I didn't have enough resistors but the software is running the whole flashing the lights in sequence idea and that works fine.

Will attempt to give it more of a go later, I think the downfall is that the pixart camera has a low field of view, so . you have to be quite far away for it to work properly. But hopefully when adding in the other 4 LEDs in the middle positions you'll be able to stand much closer.


Pictures:

 

[bobbydilley]

I've been looking through the patents for how the light gun technologies in arcade machines might work, and found this one:

https://patents.google.com/patent/US5853324A/en?q=(A63F13/219)&assignee=Namco+Ltd.

Take a look at the pictures and have a guess which game this might be referring to?



On a serious note though, these seem to be the patents related to IR tracking: https://patents.google.com/?q=(A63F13/219)

 

[winteriscoming]

I wonder if some kind of external lense could be applied to increase field of view.

With regards to brightness, is that going to be a matter of finding brighter IR LEDs or using more than one clustered together in one spot?

 

[twistedsymphony]

well the Sega setup uses 2 LEDs per module, and after talking with Ken they had special units made with 4LEDs per module which improved the range.

brightness matters, and increasing the number of LEDs will increase the brightness..

also keep in mind that LEDs themselves have a viewing angle. I'm sticking to 30-40deg LEDs for my setup.

 

[bobbydilley]

Although I'm sure you've already perfected your method for calculating the position ...

Hardly, as I mentioned in post #20 my calculations in posts #14 and #15 are problematic because I was using the wrong projection system, really they need to be thrown out and redone, but the theory in post #5 I think is still the best way to go.

I don't see much discussion of the technique used in the RAYS PCB on any forums.

While this requires much easier calculations and less complex electronics, the rest of the hardware is much more difficult to make work.
You need a specialized screen surface to reflect the dot off of, something that a normal CRT or an HDTV wont have, and then you also need permanent mounting position in relation to that screen to mount the camera. This again will be difficult for most home setups and for even most generic cabinets.

I get the impression that this system also gets upset with outside light sources and reflections, which is why most of the cabs that use it are also designed in such as way as to block outside light from the screen surface.

So while the camera and LED are cheap I think getting a good screen surface and to make this system work might make it prohibitively more challenging than a gun mounted camera with LEDs around the parameter of the screen.

Sorry to being up an old thread but thought I’d just put my findings here. I purchased the bits for a RaysIO setup and did some testing. Inside the gun is a really powerful infrared LED (not a laser diode) which is focused through the same lens as used in the photosensor versions of the guns. The LED board just fits into any normal Namco gun.

From testing with a camera with no infrared filter the light it shines is bright and clear on any surface. Have tried on walls and LCDs and it reflects well. I think this debunks the myth that you need some sort of special reflective surface for the rays IO. I think it would work in pretty much any monitor!

 

[Zebra]

I did some limited test with an IR laser plus camera solution and I found a variation in results based on the type of screen (and possibly the brand). More specifically, my LG Oled has some kind of super effective anti-reflective surface that prevents a visible reflection of any of my light sources. It worked fine on a cheaper LCDs, rear projection and front projection screens though.

My assumption is that Namco used a mirror cab for Crisis Zone for this reason. I assume the rear projection versions didn't need the mirror. It's just a guess though.

If it was just for personal use (instead of a commercial product) then there is lot of potential solutions including laminating your own clear matte coating onto some plexiglass for the cab screen covering. Perhaps those transparent rear projection window films would be effective too.