Cannot get mouseUp or MouseDown events when cursor it outside if window

Xaon420

Events like:

function TopBar:onMouseDown(b, x, y) --added to catch when the user (KINDA) leaves the main window
  IsDrag = true
  MovUI(TopBar, {left = true})
end
function win:onMouseDown(b, x, y) --added to catch when the user (KINDA) leaves the main window
  IsDrag = true
  MovUI(TopBar, {left = true})
end

do not seem to register the mouse event when the cursor it outside of the window, this is kinda annoying since the task delay when using sleep(0) or sleep(1), using sleep() in the task results in other tasks being cut-off from execution (meaning tracking cursor movement for canvas is delayed or wont register when out of window for even a ms, resulting in a situation where the user needs to click again, then messing up paint or cursor operations)

Longer story shortened; i cant detect mouse events when out of the window, If they are mouse events they should always fire and then its up to me to defer which window event fired; my suggestion is a module (like the keyboard module) that is able grab what monitor the mouse is on and its position, and button states via events (I did try building luart with a mouse module but im not familiar with c much anymore)

Samir

This behavior comes from the way the Windows API handles mouse events: mouse input is dispatched on a per-window basis.

When the cursor leaves a window (or a widget), that window no longer receives mouse events. As a result, events such as onMouseMove or mouse button notifications cannot be delivered until the cursor re-enters the window.

Xaon420

Samir Sorry for the late response, Been busy so ill try to keep it short but concise hopefully
though i have found a solution to my issue, its just not integrated in luart sadly

Binaries

im not able to follow the guide or figure much out for making working luart binaries i can get compiling dll's with luart but they just simply crash luart without any errors i can find; they are a struggle so far. though i might give it another go after i relearn some C and the inner workings of luart itself.

Request / Missing Feature

I suppose what luart is lacking is a mouse module or any stats on the Mouse (in linux its "xev").

since windows defines those devices as user input devices anyway, i can grab them from powershell and luart C Module but unsure how to make it a binary module for myself and others

I guess simply put as well; what im looking for is a way to poll the mouse via a lua call (X, Y, M1, M2, MMB) in a new native windows module, rather than relying on only ui events.
along with that in some cases the UI may not be wanted for some projects, leading to no way of getting mouse data
Polling when wanted also helps with high DPI and Polling rates (mine is the lowest it can go, 150 per ms. "Fast" movement causes the same issue with a windows the size of the monitor)

Anyway The main Example:

--! luart-extensions
local c = require "c"

local user32 = c.Library("user32.dll")

user32.GetCursorPos = "(p)I" 
user32.GetAsyncKeyState = "(i)i"

local POINT = c.Struct("ii", "x", "y") 

-- Virtual Key Codes
local VK_LBUTTON = 0x01
local VK_RBUTTON = 0x02
local States = {m1 = false, m2 = false}

while true do
    local mousePos = POINT()

    if user32.GetCursorPos(mousePos) ~= 0 then
        print(string.format("Mouse Position: X = %d, Y = %d", mousePos.x, mousePos.y))
    end

    if (user32.GetAsyncKeyState(VK_LBUTTON) & 0x8000) ~= 0 and not States.m1 then
        print("Left Mouse Button Down!")
        States.m1 = true
    elseif (user32.GetAsyncKeyState(VK_LBUTTON) & 0x8000) ~= 0 and States.m1 then
        print("Left Mouse Button Held!")
    elseif (user32.GetAsyncKeyState(VK_LBUTTON) & 0x8000) == 0 and States.m1 then
        print("Left Mouse Button Up!")
        States.m1 = false
    end
    
    if (user32.GetAsyncKeyState(VK_RBUTTON) & 0x8000) ~= 0 and not States.m2 then
        print("Right Mouse Button Down!")
        States.m2 = true
    elseif (user32.GetAsyncKeyState(VK_RBUTTON) & 0x8000) ~= 0 and States.m2 then
        print("Right Mouse Button Held!")
    elseif (user32.GetAsyncKeyState(VK_RBUTTON) & 0x8000) == 0 and States.m2 then
        print("Right Mouse Button Up!")
        States.m2 = false
    end
    sleep(100) 
end

The powershell Example that kinda got me using the C module (Which is my goto now)

MouseReader.lua Example:

local sysutils = require("sysutils")

--## Documentation
-- *local MD = ReadMouse() --reads mouse Data
-- *WriteProc() --Any string ending in \n will be sent to the powershell script
MouseData = {} --Access a Last Snapshot of MouseData from here
MouseEvent = function(MD) end --Will be Fired When any data comes from terminal (Todo, may scratch this off; kinda like an event but maybe just one callback aint all bad)


--### Program Stuff
MouseThreadPriority = 2 --from (0~X)
ReadMouse = nil

--### Main Task
local MouseTask = sys.Task(function(T)
  --Set timeout for "forever"
  T.timeout = 600000

  --## create and spawn Reader
  local process = sysutils.Process("powershell.exe", {
    "-NoProfile",
    "-NonInteractive",
    "-ExecutionPolicy", "Bypass",
    "-File", "F:\\LuaWorkspace\\L2.2\\mouse.ps1"
  })
  process = sysutils.Process("powershell.exe -NoProfile -ExecutionPolicy Bypass -File F:\\LuaWorkspace\\L2.2\\mouse.ps1", false, true)
  print(process:spawn())

  --## process Write Task
  local TmpTsk21
  local ProcQ, WriteFlag = {}, 0
  local WriteThread = sys.Task(function()
    print("[THREAD] WriteThread Started!")
    while true do
      if #ProcQ >= 1 and WriteFlag == 0 then --can async this now if needed for some extra lil performance but meh
        --print("POP: ", #ProcQ, string.gsub(ProcQ[1], "\n", ""))
        local pop = ProcQ[1]
        table.remove(ProcQ, 1)
        WriteFlag = #ProcQ
        process:write(pop) --can add the \n here if i want, this way its par to a print func
        WriteFlag = 0
      else
        sleep()--let the vm decide (To the upper note; also lets those tasks finish better bc of this so its supported for it either way)
      end
    end
  end)
  WriteThread.priority = MouseThreadPriority-2 --lower priority bc this shouldnt take alot of headroom
  
  --## Internal Write Handle to the ps1, Yea Sloppy but is working up to 1~2ms reads depending on task usage/config
  local WriteProc = sys.TaskFactory(function(Msg)
    while (WriteFlag >= 1 and #ProcQ <= 32) do if #ProcQ >= 16 then sleep(20*#ProcQ) else sleep() end end --35ms timeout or Auto
    if #ProcQ >= 32 then
      --table.remove(ProcQ, #ProcQ) --if overburdened then just remove some recent items
      return "StackLimited"
    end
    ProcQ[#ProcQ+1] = Msg --write to stack
  end)

  --## Globalized non-Async function to Call MouseData (may need to fix for Higher Polling within 2ms)
  local TReadMouse = function()
    WriteProc("ReadMouse\n")
    sleep()--smol dly, helps with the ps script
    return MouseData
  end
  ReadMouse = TReadMouse

  --Handle Output and Parse for string Manipulation (outputs to MouseData, left, right, middle are on 3~5 index's, 1&2 are pos at poll time)
  function process:onRedirect(data)
    async(function()
      local m = tostring(data):match("([^\r\n]+)%s*$")
      --print("Ps1:", m)
      if m:match("MouseData:") then
        --print("MouseData Updated")
        local tmp = string.gsub(m, "MouseData:", "")
        MouseData = splitTbl(m, ",")
      else
        --MouseData = {}

      end
      data = nil
    end)
  end

  --## start ProcMsg Handler
  WriteThread()

  --##Poll mouse Loop (Keeps the ps1 alive, so we can poll when needed)
  while true do
    WriteProc("KeepAlive\n")--keeps the ps1 alive, as does ReadMouse; but this is less intensive (ps1 has a 8s window)
    sleep(300)
    local MD = ReadMouse()
    print("MousePoll:", MD[1], MD[2]) --this is how you ge the data, it will be in MouseData if sucessful
  end
end)



--Ignore (for another Lib i have)
MouseTask.timeout = -600000
MouseTask.priority = MouseThreadPriority-1
function StartTask(T)
  MouseTask(T or t)
end


--## Start Mouse Task 
StartTask(MouseTask)
sleep(3000)
print("MousePoll: ", ReadMouse())

--await and then close the proccess, if closing luart via taskmanager or vscode the old proccess will kepe running
await(t)
proccess:close()--not needed as the PS will close after roughly 8s

Powershell Mouse Reader Dependency:

Add-Type @"
using System;
using System.Runtime.InteropServices;

public class MouseState {
    [DllImport("user32.dll")]
    public static extern bool GetCursorPos(out POINT lpPoint);

    [DllImport("user32.dll")]
    public static extern short GetAsyncKeyState(int vKey);

    public struct POINT {
        public int X;
        public int Y;
    }
}
"@

# Start watchdog thread
$JobName = "Test"
$heartbeatFile = "$env:TEMP\mouse_keepalive.txt"
Set-Content $heartbeatFile (Get-Date)
$lastKeepAlive = Get-Date (Get-Content $heartbeatFile)
$elapsed = ((Get-Date) - $lastKeepAlive).TotalMilliseconds

$Job = Start-Job -Name $JobName -ArgumentList $heartbeatFile -ScriptBlock {
    param($heartbeatFile)

    while ($true) {
        Start-Sleep -Milliseconds 10

        $lastKeepAlive = Get-Date (Get-Content $heartbeatFile)
        $elapsed = ((Get-Date) - $lastKeepAlive).TotalMilliseconds

        Write-Host "Running: $elapsed"

        if ($elapsed -gt 8000) {
            Write-Output 0
            break
        }
    }
}


while ($true) {

    # Check if watchdog finished without blocking input
    if ($Job.State -eq "Completed") {
        $Quit = Receive-Job $Job
        Write-Host "IsGoingToExit: $Quit"
        break
    }


    # Read input safely from either console or redirected pipe
    $command = $null

    if (-not [Console]::IsInputRedirected) {

        if ([Console]::KeyAvailable) {
            $command = [Console]::ReadLine()
        }

    }
    else {

        if (-not [Console]::In.EndOfStream) {
            $command = [Console]::In.ReadLine()
        }

    }


    # Process command if one was received
    if ($null -ne $command) {

        if ($command -eq "ReadMouse") {

            $elapsed = ((Get-Date) - (Get-Date (Get-Content $heartbeatFile))).TotalMilliseconds
            Set-Content $heartbeatFile (Get-Date)

            $pos = New-Object MouseState+POINT
            [MouseState]::GetCursorPos([ref]$pos)

            $left = ([MouseState]::GetAsyncKeyState(0x01) -band 0x8000) -ne 0
            $right = ([MouseState]::GetAsyncKeyState(0x02) -band 0x8000) -ne 0
            $middle = ([MouseState]::GetAsyncKeyState(0x04) -band 0x8000) -ne 0

            [Console]::WriteLine("MouseData:$($pos.X),$($pos.Y),$left,$right,$middle")
            [Console]::Out.Flush()
        }

        elseif ($command -eq "KeepAlive") {

            $elapsed = ((Get-Date) - (Get-Date (Get-Content $heartbeatFile))).TotalMilliseconds
            Set-Content $heartbeatFile (Get-Date)

            [Console]::WriteLine("KeepAlive: $elapsed ms")
            [Console]::Out.Flush()
        }

        else {
            [Console]::WriteLine($command)
            [Console]::Out.Flush()
        }
    }

}

Remove-Job $Job

Samir

Your c FFI approach seems correct adn the simplest approach.
Just wrap the while true... do ... end loop in a Task so you can run it concurrently with the ui Task

The Powershell way seems to hacky and complex I think.

Btw, for the dll, you mean you can't build a third party binary module for LuaRT ? You should see as an example the src/modules/ini/src/ini.c to show you how the simple module ini is built. You can reuse the Makefile too with its nmake.bat helper too