Some poker sites have code built into their clients that allow them to display a captcha challenge when the server requests it (which likely is manually triggered by the security team after reports from other users, or automatically if your playing behavior shows suspicious metrics as detailed by @Grinch91). And some also do track mouse movement to see if it moves in a natural fashion like a human would.
I've never been shown one personally (I don't use a bot however :P) but I've seen screenshots and also some files/symbols with very informative names while skimming through the program files and the binary code of the executables.
Captchas aren't bulletproof though, and are likely only being used as another indication if a particular account might be operated by a bot. Services exist (in third-world countries, where labor is cheap) where you pay a cent fee to have a captcha solved for you by a real human. They provide APIs to integrate their service into automated workflows.