Bluetooth Filter Driver for DS3-compatibility - research notes



  • HCK had its run over night on the profile driver and look how happy it is πŸ˜…

    76f129b3-e1ce-46f2-bbaf-8504840ef178-image.png

    Next I have to polish and test the filter driver and extend it with a few management features (like turning it on or off from user-land).



  • @nefarius this looks awesome! Really looking forward to seeing this in production! Is there anything we as a community can do or provide to assist your work? Beta testing, experiment with various BT dongles/pads, or just show interest in your product? πŸ™‚ What kind of licensing are you thinking about once this is ready? Propriety paid/free, open source and all its variations, dual - community version open source (without binaries) VS paid WHQL certified?



  • @Locksmith pardon the delay, I wanted to answer this in detail as it contains a lot of interesting and valid questions.

    Community involvement

    I will be getting to a stage where the full stack can be tested and I would greatly appreciate it if I get a few owners of those obviously "fake" and aftermarket DualShock "PANHAI" controllers since I can't order them in my country and back in SCP-days a rather large demographic was interested in getting them to work as they typically only cost around 10$. I'm not a fan of those personally but if they work on the stack with ease, be my guest πŸ˜‰ As I own the genuine Sony hardware I can guarantee a 100% compatibility for those obviously. Just a heads-up: I won't give out production-signed binaries to beta testers anymore though as that's irresponsible. So anybody interested would be required to put his or her machine into test mode. Usually not a big deal but I've heard that a few anti-cheat and DRM solutions out there won't let you launch certain games anymore until reverted.

    Dongle compatibility shouldn't be much of an issue anymore as the vendor software stack is kept in place and isn't touched anymore (unlike SCP) so e.g. Intel Wireless cards will keep operating with the Intel vendor driver. Tests welcome though nonetheless.

    Last but not least I ofc. welcome interest in my post-apocalypticSCP shenanigans πŸ˜‰ I've launched into this mess out of personal motivation; getting comfortable with yet another kernel-mode driver, the Windows Bluetooth Driver Interface, WHQL-fun and maybe finally getting a finished plug-and-play solution so I can visit my couch again and play Castlevania πŸ˜† A lot of what's happening here currently is straight out research and development. As a lot of people are apparently not understanding (or accepting) what "work in progress" means I keep "advertising" for the forum etc. to a minimum as long as there's not enough resources for me to research, develop, give support, regularly update news etc. This doesn't mean I want to stay in my own bubble here, I welcome anybody interested but in return I expect acceptance if things don't quite go according to plan.

    Licensing

    Despite a few "hiccups" I've had with larger corporations taking my stuff being a bit clumsy in the past I'm still a big advocate and believer in open-source. First and foremost because I'd probably not be in this "business" of dealing with gaming peripherals if the mad lad creating SCP hadn't open-sourced his or her work plus all of the other fabulous projects and samples I profited of by studying the sources. This also includes non-Windows sources like bits and pieces in the Linux kernel or Arduino projects. So I'll definitely open-source this stack as well once I tag it production-ready.

    Now one might ask why this time I didn't go for GitHub from the very beginning but use a private Git repository instead. It makes developing the back-doors so much easier! πŸ˜† Jokes aside, once again this is another measure I had to take to protect myself from idiocy. What I've noticed over the years is that non-developers who find something interesting on GitHub usually ignore the README and other helpful resources completely, get confused because there is no "download EXE here now" button and then come haunt and annoy me and my team with questions that could've been avoided from the very start. I have no problem with answering questions but if I get used as a search engine replacement I take that as a personal offense as you clearly show me that you don't value my time. Phew, that went sour fast πŸ˜…

    Back to licensing. With upcoming releases I'm gonna include an EULA to both protect myself (you know, warranty, haha) and probably place some restrictions on it regarding commercial use. Maybe dual-licensing it. Again, there are reasonable motivations behind it. To private individuals I wanna ensure all the freedom and openness as possible as that's the target demographic. Corporations though are dumb. What do I mean by that? Well, just forking a driver, re-branding and releasing it (not that this has happened before, huehuehue) is a horrible practice. Drivers need maintenance and the knowledge on how to maintain them properly. If they cause an issue on an end-users machine it needs to be diagnosed and addressed. Can they provide that? No, they'll conveniently send them to me who's expected to give free support and fix everything. Nope, get fucked, GPL and other restrictions it will be for you then. I haven't settled on the best solution there yet but I'm working on ideas that benefit all sides as much as possible.

    Regarding WHQL... Binaries without it aren't really worth it nowadays so it's either just the sources and have fun or a properly signed binary. Could I sell it? Eh. Would it sell? Eh. Would I take a few Schekels if offered? Oh, absolutely, the bills haven't stopped 🀣 I need to figure this out. Bring back donations? Eh. Maybe they'll flourish again if a finished attractive product is released? So many questions, so little time. We'll figure it out πŸ˜€

    I think I've got everything out I wanted to address, pardon the wall of text πŸ˜…

    Cheers



  • Meh, HCK tests on Windows 8.1 do a bit more in-depth Bluetooth testing it seems and it's complaining that I've left out some I/O handlers I thought I don't need. Well, can't argue against the machine so back to hacking and fixing πŸ’€

    34182df1-4525-4f2f-9bdc-9941ffabf32f-image.png



  • There we go! πŸ˜ƒ

    e8e9995d-4abf-45e6-b82d-94e61c28e311-image.png



  • Another night of wasting electricity done but the results at least are satisfying 😁

    e4024742-8380-48fd-8cba-dad69cfe6a90-image.png

    Next I'll test with children connected. Not sure how it'll react since the power and sleep tests will most certainly disconnect them from the Bluetooth host but we'll see!



  • I was bitching a lot about the Hardware Lab Kit and debugging utilities in general in the past but to perfectly honest; the more I become familiar with them and overcome their quirks the more I appreciate the feedback they provide.

    Like for example; I've tried to connect a DS3 while the profile driver is running under Driver Verifier (which in short means that a strong rule-set applies triggering bugs that might have gone under) on Windows 8.1 and it crashed with:

    KMODE_EXCEPTION_NOT_HANDLED (1e)
    This is a very common bugcheck.  Usually the exception address pinpoints
    the driver/function that caused the problem.  Always note this address
    as well as the link date of the driver/image that contains this address.
    Arguments:
    Arg1: ffffffff80000003, The exception code that was not handled
    Arg2: fffff801f61cd3c0, The address that the exception occurred at
    Arg3: 0000000000000000, Parameter 0 of the exception
    Arg4: 7efefefefeff3252, Parameter 1 of the exception
    

    Once in WinDbg with the symbols loaded we get a nice "stack trace":

    STACK_TEXT:  
    ... : nt!KeBugCheckEx
    ... : nt!KiFatalExceptionHandler+0x22
    ... : nt!RtlpExecuteHandlerForException+0xd
    ... : nt!RtlDispatchException+0x1a5
    ... : nt!KiDispatchException+0x18d
    ... : nt!KiExceptionDispatch+0xc2
    ... : nt!KiBreakpointTrap+0x2dc
    ... : nt!DbgBreakPoint+0x1
    ... : Wdf01000!FxIoTargetSendIoctl+0x282da
    ... : Wdf01000!imp_WdfIoTargetSendIoctlSynchronously+0x48
    ... : BthPS3!L2CAP_PS3_HandleRemoteConnect+0x16a [e:\development\gogs\bthps3\bthps3\l2cap.c @ 49] 
    ... : BthPS3!BthPS3_IndicationCallback+0x9c [e:\development\gogs\bthps3\bthps3\bluetooth.c @ 552] 
    ... : bthport!L2CapInt_ProcessL2capConnectReq+0x40b
    ... : bthport!L2CapInt_ProcessSignallingPacket+0x45f
    ... : bthport!L2CapInt_ProcessReadBip+0x13a
    ... : bthport!HCI_ProcessAclReadBip+0x661
    ... : bthport!HCI_ProcessAclRead+0x2a8
    ... : bthport!HCI_ProcessMpBip+0x92
    ... : bthport!BTHPORT_RecvMpBip+0x41
    ... : BTHUSB!BthUsb_ReadTransferComplete+0x18d
    ... : BTHUSB!UsbWrapWorkRoutine+0x18d
    ... : BTHUSB!UsbWrapInterruptReadComplete+0x1d3
    ... : nt!IovpLocalCompletionRoutine+0x174
    ... : nt!IopfCompleteRequest+0x2ee
    ... : nt!IovCompleteRequest+0x1d7
    ... : Wdf01000!FxRequest::CompleteInternal+0x23c
    ... : Wdf01000!imp_WdfRequestComplete+0x8c
    ... : BthPS3PSM!UrbFunctionBulkInTransferCompleted+0x107 [e:\development\gogs\bthps3\bthps3psm\filter.c @ 233] 
    ... : Wdf01000!FxRequestBase::CompleteSubmitted+0x459
    ... : Wdf01000!FxIoTarget::_RequestCompletionRoutine+0x162
    ... : nt!IopUnloadSafeCompletion+0x49
    ... : nt!IovpLocalCompletionRoutine+0x174
    ... : nt!IopfCompleteRequest+0x2ee
    ... : nt!IovCompleteRequest+0x1d7
    ... : USBPORT!USBPORT_Core_iCompleteDoneTransfer+0xa02
    ... : USBPORT!USBPORT_Core_iIrpCsqCompleteDoneTransfer+0x21c
    ... : USBPORT!USBPORT_Core_UsbIocDpc_Worker+0x238
    ... : USBPORT!USBPORT_Xdpc_Worker_IocDpc+0x1fe
    ... : nt!KiExecuteAllDpcs+0x1b0
    ... : nt!KiRetireDpcList+0xd7
    ... : nt!KiIdleLoop+0x5a
    

    Alrighty, so line 49 in the function L2CAP_PS3_HandleRemoteConnect in l2cap.c of the driver BthPS3 is triggering the fault. Let's see what's there:

    aa26ddef-fe79-44b6-86a7-b926f86433be-image.png

    Ah, an inline function! What does it contain that might bomb? I have a suspicion (mostly because it's hinted in the stack trace πŸ˜‰ )...

    1fb3a003-67bc-40b5-b66e-d24d26a82546-image.png

    Let's have a look at the docs of WdfIoTargetSendIoctlSynchronously and check out the requirements section:

    IRQL PASSIVE_LEVEL

    Hm, but our L2CAP callback is invoked at PASSIVE_LEVEL, right? Let's throw in some tracing and have another run:

    2019/03/10-10:46:19.023	TRACE_LEVEL_VERBOSE	BthPS3_IndicationCallback Entry (Low (0x00))
    2019/03/10-10:46:19.023	TRACE_LEVEL_VERBOSE	L2CAP_PS3_HandleRemoteConnect Entry (Low (0x00))
    2019/03/10-10:46:19.038	TRACE_LEVEL_VERBOSE	BthPS3_IndicationCallback Entry (DPC (0x02))
    2019/03/10-10:46:19.038	TRACE_LEVEL_VERBOSE	L2CAP_PS3_HandleRemoteConnect Entry (DPC (0x02))
    

    Hey! 😠 The second call isn't at PASSIVE_LEVEL, that's DISPATCH_LEVEL! How dare you! Oh wait, the docs, check the docs:

    IRQL Developers should code this function to operate at either IRQL = DISPATCH_LEVEL (if the callback function does not access paged memory), or IRQL = PASSIVE_LEVEL (if the callback function must access paged memory)

    Ouch! How evil! πŸ˜… So that can totally happen and !wdfkd.wdflogdump gives clear insights:

    1: kd> !wdfkd.wdflogdump BthPS3
    ...
    37: FxVerifierCheckIrqlLevel - Called at wrong IRQL; at level 2, should be at level 0
    ---- end of log ----
    

    Right, so this would've fallen under the radar if it didn't bomb in the lab. Thanks, WHQL 😝 Onward with the fixes!



  • How future proof is this filter solution? Do you think it is possible for Windows to make such a change in a update that could break it or you think that is unlikely or just straight up impossible unless windows change in a fundamental way?

    Sorry if this is no place to ask or if my question does not make much sense. I am no developer/coder in any meaningful way, but I know just enough about coding to understand your posts here and I find it really interesting to know how this actually works. I would really appreciate to hear your thoughts on this with some insight on why and how windows could or couldn't break this in the near future.
    Also, thanks for your work on the ScpToolKit and on maintaining this posts on the development! I find myself refreshing this page many times a day just to see your explanations on how you are making your way through this.



  • @kirian said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    How future proof is this filter solution? Do you think it is possible for Windows to make such a change in a update that could break it or you think that is unlikely or just straight up impossible unless windows change in a fundamental way?

    Heya! This is actually the most future-proof you can get. Apart from the filter driver which proxies the forbidden PSMs the profile/bus driver uses a standard DDI (Device Driver Interface) provided by Microsoft which exists since (at least) Windows XP and is documented fairly well (see here and here). Microsoft is the grand master of backwards compatibility so I expect this to last quite a fair bit without the need of updates once it's deemed stable.

    @kirian said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Sorry if this is no place to ask or if my question does not make much sense. I am no developer/coder in any meaningful way, but I know just enough about coding to understand your posts here and I find it really interesting to know how this actually works. I would really appreciate to hear your thoughts on this with some insight on why and how windows could or couldn't break this in the near future.

    Perfectly fine to ask here, this thread isn't meant to be only fed by myself, it's just my chaotic attempt of Rubber Duck Debugging and leaving a trail of insights on this "black magic" in kernel land πŸ˜‰ Research and development as I like to call it.

    I mean as far as the profile driver goes since it's a common practice to write profile drivers (well, probably not anymore because literally any standard device has one already) and the API is "official" and not some binary patching nonsense I'm pretty confident that this will outlive at least SCP 😝

    As for the PSM proxy... That's a bit of a risky move because I'm shipping around a function that was probably designed with a meaningful goal in mind and here I walz in and build a bridge across the forbidden river πŸ˜‰ We will see, I guess πŸ˜‡

    Also this method is far more likely to pass WHQL as I don't have to run the whole test battery directly against the USB device itself since the original driver is kept in place. With AirBender/WireShock I'd probably get a few dirty looks when I submit an INF file with a couple of dozen Hardware IDs which aren't mine to the Microsoft Portal πŸ˜‰ Plus the user gets to keep all of their other Bluetooth devices connected, isn't that lovely πŸ‘

    @kirian said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Also, thanks for your work on the ScpToolKit and on maintaining this posts on the development! I find myself refreshing this page many times a day just to see your explanations on how you are making your way through this.

    Glad to hear I'm not only feeding my echo chamber πŸ˜† Yeah, SCP's kind of my Dark Ages, it changed me in a lot of ways. Thanks for sticking around and showing interest πŸ˜ƒ

    Cheers



  • Hm, disabling/shutting down the profile device causes crashes due to referencing freed memory. Next challenge is to trace along the whole connection/disconnection state machine and identify race conditions and whatnot πŸ˜ͺ



  • Let's welcome this new week by finishing a major bug fix: properly hardening the shutdown sequence to disconnect remote devices and free resources afterwards 🀠

    972d30ca-3706-4250-832d-f6d3d84ad145-image.png

    Now entering hibernation πŸ’€



  • @nefarius said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Now entering hibernation πŸ’€

    I really liked that one πŸ˜‚ I registered to the forum just to reply to this...

    Now getting serious, I've been following this topic, let me tell you that is awesome. I was a programmer myself when I was young, like 20 years ago, nothing like this, mostly enterprise application, databases and stuff, so I have a fair idea of the whole development process thing. This thread has been very interesting to follow, I didn't know how many things got involved in developing drivers and stuff related to hardware, very interesting.
    Also, your previous post about hunting the bug related to the crash when device got disabled, it got me thinking in other things that could happen that could be related...

    What happens when

    1. The controller is moved out of range of the bluetooth receiver
    2. The bluetooth receiver is disabled in the Device Manager (eg. if the receiver is embedded in a wifi card like the Intel ones on laptops)
    3. The bluetooth receiver is disconnected from the host (eg. the receiver it's a USB dongle type)
    4. If the controller get connected to host via USB cable while operating/connected via Bluetooth protocol, do it still operates or got it disconnected???
    5. Does the controller start to charge its internal battery when is connected to host via USB cable?

    I'm sure you already considered this scenarios, but I got curious about the answer to this events.
    Anyway, let thank your for your effort in this, I really appreciate it.
    It's really amazing to see that somebody got the time, knowledge and motivation to write this. In other OS like linux, this works out-of-the-box, also on NVidia Shield TV, but on windows it's disappointing to have to resort to third-party solutions to be able use our gamepads properly.

    Also, what is the HARDWARE id that get reported to OS when the PS3 controllers are connected via this driver?



  • @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Now getting serious, I've been following this topic, let me tell you that is awesome. I was a programmer myself when I was young, like 20 years ago, nothing like this, mostly enterprise application, databases and stuff, so I have a fair idea of the whole development process thing. This thread has been very interesting to follow, I didn't know how many things got involved in developing drivers and stuff related to hardware, very interesting.

    Welcome, mate πŸ™‚ I've been trying to steer my career towards programming as a full-time profession but so far that hasn't really worked so I went back to doing everything the way I like and devote time to device driver development. Not my loss so far 😝 I struggled a lot in the past while developing the other projects and found this style of "blogging" to be a good way to both reflect on what I've been doing and as a reference and timeline for myself and other people interested. Sounds like it's a good strategy.

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Also, your previous post about hunting the bug related to the crash when device got disabled, it got me thinking in other things that could happen that could be related...
    What happens when

    Thanks for participating, let's go through the points one by one:

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    The controller is moved out of range of the bluetooth receiver

    Then after a certain timeout the host driver (the one which controls the Bluetooth host device, a.k.a. "stock driver") will initiate a disconnect sequence and notifies the profile driver that the device is gone and now my clean-up code takes over. So far this case is handled properly. Should test it though πŸ€”

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    The bluetooth receiver is disabled in the Device Manager (eg. if the receiver is embedded in a wifi card like the Intel ones on laptops)

    That's a typical power down event and since the profile driver is a child of the bthport.sys which then gets unloaded it receives shutdown indication as well and needs to dispose all connections and free memory before unloading the profile driver. So far this is handled already.

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    The bluetooth receiver is disconnected from the host (eg. the receiver it's a USB dongle type)

    This case is called "surprise removal" and is also handled. When the parent is gone, the whole stack gets demolished. The profile children (PDOs) get removed, the profile driver enters clean-up and unloads, then the stock drivers unload. This is also implemented and tested.

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    If the controller get connected to host via USB cable while operating/connected via Bluetooth protocol, do it still operates or got it disconnected???

    This is in fact an absurd case because the SIXAXIS/DS3 does not disconnect from Bluetooth when connected to USB while also connected wireless. That's a scenario I haven't tackled yet ❗

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Does the controller start to charge its internal battery when is connected to host via USB cable?

    Yes, despite all the BS and false information you find on the web there is no special driver required for the controller to charge via a standard 500 mA USB outlet. It also charges with a simple mobile phone charger, if it doesn't it has a hardware issue, not a software one. It does report battery charge level via software though on both USB and Bluetooth.

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    I'm sure you already considered this scenarios, but I got curious about the answer to this events.

    I try to think of all aspects, especially because I wanna get this through WHQL so the quality and robustness has to be stellar πŸ˜‰

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Anyway, let thank your for your effort in this, I really appreciate it.
    It's really amazing to see that somebody got the time, knowledge and motivation to write this. In other OS like linux, this works out-of-the-box, also on NVidia Shield TV, but on windows it's disappointing to have to resort to third-party solutions to be able use our gamepads properly.

    Thanks, I might have just slipped into insanity without noticing πŸ˜† Seriously though, it's been quite the challenge getting this and real-life obstacles handled in one go but as I'm adapting my life to support this stuff I might be able to keep this rodeo going until production-ready πŸ˜… Linux has the advantage of the open kernel and contributors have added the Sony-specific customization a long time ago. On Windows you need to play after the rules of Microsoft. And dance with the devil in kernel-land where the forbidden fruits grow and magic can be found even to this day!

    @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Also, what is the HARDWARE id that get reported to OS when the PS3 controllers are connected via this driver?

    I created some custom, GUID-based Hardware IDs the function drivers will use in the future.

    Hope I got everything, cheers!



  • @nefarius said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    Hope I got everything, cheers!

    Wow, I'm really impressed... I'm pretty sure that I'm going to use this driver when it get released intro production.
    I only regret having bought a Mayflash Wireless Adapter like 3-4 months ago to use my SIXAXIS/DS3 wireless without having to sacrifice the bluetooth receiver on a PC that I use to play retro games. πŸ˜… I needed to use a wireless headphones to play at night, so that's why I needed the bluetooth device on the host working with standard drivers (not dedicated to just the controller), to be able to connect the headphones... you know, I didn't want the wife to be mad at me for being playing games at night.... πŸ˜’

    I forgot to ask... how is going to be the process to pair the controller with the host receiver? I don't remember exactly, but in the old days of ScpToolkit and Motioninjoy there was a small utility to set the host's bluetooth mac to which the controller should connect when the user pressed the PS button... please correct me if I'm wrong... how is going to be now?



  • @pnkiller78 said in Bluetooth Filter Driver for DS3-compatibility - research notes:

    I forgot to ask... how is going to be the process to pair the controller with the host receiver? I don't remember exactly, but in the old days of ScpToolkit and Motioninjoy there was a small utility to set the host's bluetooth mac to which the controller should connect when the user pressed the PS button... please correct me if I'm wrong... how is going to be now?

    That's still the same; you send a single request to the device via USB updating the host MAC address it shall connect to. This can be done via SCP, FireShock or even WinUSB and a bit of custom code. No biggie.

    I'll provide a tool for that.



  • Ugh, I'm on a hunt. A bug hunt. And it's always issues I've introduced myself πŸ˜†

    2: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************
    
    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: ffffe000c63b0dd4, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
    Arg4: fffff800ccac6434, address which referenced memory
    

    DRIVER_IRQL_NOT_LESS_OR_EQUAL is the most misleading bugcheck there is because the IRQL has little to do with this particular case, it's just me accessing freed memory:

    ca8ee713-3e75-4251-8632-5c2c9e29e881-image.png

    A classic 😁



  • Hello @nefarius,
    What wonderful work you have been doing!

    If you could put the code on github, (possibly just your β€œsrc” ans β€œinclude” directories {and licence of course}, so that noobs won’t compile and break their system.). It would allow some of us to have a more in-depth read-through of your code.

    Otherwise, keep up the amazing work; I look forward to seeing the code in person.

    Cam.



  • @da2ce7 Greetings. The plan is indeed to move to GitHub once the project is mature and stable enough. Right now it will stay private 😁


Log in to reply