C# Bejeweled 3 bot - part 3 - handling mouse and keyboard

Handling Mouse

Making mouse move and click on stuff in C# is probably the easiest thing I did in my life. Here is the class:

using System.Drawing;
using System.Windows.Forms;
using System.Runtime.InteropServices;

namespace BejeweledBot
{
    class VirtualMouse
    {
        [DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
        public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);
        private const int MOUSEEVENTF_LEFTDOWN = 0x02;
        private const int MOUSEEVENTF_LEFTUP = 0x04;
        private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
        private const int MOUSEEVENTF_RIGHTUP = 0x10;

        private Cursor Cursor;

        public VirtualMouse()
        {
            this.Cursor = new Cursor(Cursor.Current.Handle);
        }

        public void Move(int X, int Y)
        {
            Cursor.Position = new Point(X, Y);
        }

        public void Click()
        {
            mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, (uint)Cursor.Position.X, (uint)Cursor.Position.Y, 0, 0);
        }
    }
}

With this class, if you call Move(…) followed by a Click() you will have your mouse and raise click events on any point on the screen.

Just after testing the mouse behaviour I saw a major flaw in my design: when your program takes over the mouse control and clicks on things twice every 300 ms and on top of that because your program is not in focus anymore (you are doing some heavy work with your mouse on your web browser after all) you can’t really handle the typical key events on your form (and you accidentally start calling your friends on skype), which brings us to:

(Globally) handling keyboard events

Before you start thinking of me in a high manner, let’s give credit where it’s due. And it’s due to Stephen Toub for writing the article “Low-Level Keyboard Hook in C#”. Using his solution and a bit of hackery I started forwarding global key press events to my form, just like this:

using System;
using System.Diagnostics;
using System.Windows.Forms;
using System.Runtime.InteropServices;

namespace BejeweledBot
{
    /// <summary>
    /// Based on https://blogs.msdn.microsoft.com/toub/2006/05/03/low-level-keyboard-hook-in-c/
    /// </summary>
    class KeyboardIntercept
    {
        public EventHandler<KeyEventArgs> KeyIntercepted;

        private const int WH_KEYBOARD_LL = 13;
        private const int WM_KEYDOWN = 0x0100;
        private static LowLevelKeyboardProc _proc = HookCallback;
        private static IntPtr _hookID = IntPtr.Zero;

        public KeyboardIntercept()
        {
            instance = this;
        }

        public static void Hook()
        {
            _hookID = SetHook(_proc);
        }

        private static KeyboardIntercept Instance { get { return instance; } }
        private static KeyboardIntercept instance;

        public static void Unhook()
        {
            UnhookWindowsHookEx(_hookID);
        }

        private static IntPtr SetHook(LowLevelKeyboardProc proc)
        {
            using (Process curProcess = Process.GetCurrentProcess())
            using (ProcessModule curModule = curProcess.MainModule)
            {
                return SetWindowsHookEx(WH_KEYBOARD_LL, proc,
                    GetModuleHandle(curModule.ModuleName), 0);
            }
        }

        private delegate IntPtr LowLevelKeyboardProc(int nCode, IntPtr wParam, IntPtr lParam);

        private static IntPtr HookCallback(int nCode, IntPtr wParam, IntPtr lParam)
        {
            if (nCode >= 0 && wParam == (IntPtr)WM_KEYDOWN)
            {
                int vkCode = Marshal.ReadInt32(lParam);
                Instance.onKeyIntercepted((Keys)vkCode);
            }

            return CallNextHookEx(_hookID, nCode, wParam, lParam);
        }

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        [return: MarshalAs(UnmanagedType.Bool)]

        private static extern bool UnhookWindowsHookEx(IntPtr hhk);

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr GetModuleHandle(string lpModuleName);

        private void onKeyIntercepted(Keys key)
        {
            KeyIntercepted?.Invoke(this, new KeyEventArgs(key));
        }
    }
}

To forward the key presses to my form it’s enough that I subscribe to the KeyIntercepted event:

KeyboardIntercept keyboardIntercept = new KeyboardIntercept();
keyboardIntercept.KeyIntercepted += new EventHandler<KeyEventArgs>(Form_KeyDown);

In my Form_KeyDown(…) method I check if the pressed key was Escape and if it was then I stop the main timer so that I can move my mouse by myself.

The end!

This was my last post about making Bejeweled 3 bot in C#. Please feel free to fork it, change it, do your own implementation etc. You will find the repo here.

Mateusz Sadowski

Mateusz Sadowski
Mat is a Robotics consultant and the author of Build Mobile Robots with ROS 2 LiveProject series.

Remote Robotics Consulting - A Five-Year Retrospective

In this blog post, I will highlight some of the updates since my last blog and offer some advice that I hope will be useful for anyone looking to get into technical consulting. Continue reading

I Created a Project-based ROS2 Course

Published on July 05, 2022

IMUs and LiDARs - Not Uncommon Pitfalls

Published on February 24, 2022