Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifier keys don't really seem to work #47

Open
nnmrts opened this issue Dec 13, 2021 · 0 comments
Open

Modifier keys don't really seem to work #47

nnmrts opened this issue Dec 13, 2021 · 0 comments

Comments

@nnmrts
Copy link

nnmrts commented Dec 13, 2021

Description

Modifier keys don't really work like they should on MacOS. Maybe this is only an issue with MacOS Monterey, I don't think it is though. I can't really "describe" the bug in detail because I have no clue what's really going on. All I know is that using toggleKey and then type with a single letter really just inserts that letter as text and autopilot will completely ignore that modifier keys are ignored. However, I can compare autopilot's behaviour with cliclick's.

Reproduce

  1. Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
  2. Run this:
    deno repl --no-check --unstable
  3. Then enter this and press enter:
    import AutoPilot from "https://deno.land/x/autopilot/mod.ts";
    
    const sleep = async (milliseconds) => new Promise(resolve => setTimeout(resolve, milliseconds));
    
    await sleep(5000);
    
    await pilot.toggleKey("meta", true);
    await pilot.toggleKey("shift", true);
    await pilot.type("g");
    await pilot.toggleKey("meta", false);
    await pilot.toggleKey("shift", false);
  4. You now have five seconds to switch to your browser again and click on that input field.
  5. This is the result:
    Bildschirmfoto 2021-12-13 um 06 40 55
  6. Install cliclick and run this, then click on the input field on the website again:
    cliclick w:5000 kd:cmd,shift t:g ku:cmd,shift
  7. This is the result:
    Bildschirmfoto 2021-12-13 um 06 55 34

Expected Behaviour

While modifier keys somehow work individually, they don't work like expected in most cases I experienced. For example, all hotkey combinations that include a modifier key and any ordinary letter, like the typical +C to copy, just don't work in my experience. To give you one more example, on MacOS, whenever an "open file" dialog is present, you can use ++G to open a textbox allowing you to enter the path to a folder. Using cliclick for this just works, while autopilot doesn't, and instead just presses G.

Desktop

  • OS: MacOS Montery 12.1 Beta (21C5045a)
  • Browser Chrome
  • Version 0.4.0

Additional context

Going through the issues of this repository, I assume the root of this issue going unnoticed is toggleKey only allowing specific keys for some reason. Looks like up until version 0.4.0, hotkey stuff did work, by using toggleKey. Note that I haven't found any evidence of the combination toggleKey(modifier, true); type(letter); toggleKey(modifier, false); ever working. If one uses ordinary letters with toggleKey today, everything crashes and rust panicks:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error("unknown variant `c`, expected one of `f1`, `f2`, `f3`, `f4`, `f5`, `f6`, `f7`, `f8`, `f9`, `f10`, `f11`, `f12`, `f13`, `f14`, `f15`, `f16`, `f17`, `f18`, `f19`, `f20`, `f21`, `f22`, `f23`, `f24`, `leftarrow`, `uparrow`, `rightarrow`, `downarrow`, `insert`, `delete`, `home`, `end`, `pageup`, `pagedown`, `tab`, `backspace`, `enter`, `escape`, `space`, `meta`, `alt`, `control`, `shift`, `capslock`", line: 1, column: 10)', src/lib.rs:122:1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5

Backtrace with RUST_BACKTRACE=1:

stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: _toggle_key
   4: _ffi_call_unix64
   5: _ffi_call_int
   6: deno_ffi::ffi_call
   7: deno_core::ops_json::op_sync::{{closure}}
   8: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
   9: deno_core::ops::OpTable::route_op
  10: <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn
  11: __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
  12: __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
  13: __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
fatal runtime error: failed to initiate panic, error 5

Backtrace with RUST_BACKTRACE=full:

stack backtrace:
   0:        0x1144675d1 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hec762383110a7685
   1:        0x11447f3bb - core::fmt::write::h2d5ecb4b9764759c
   2:        0x114465c0a - std::io::Write::write_fmt::h9d7d3ae333151289
   3:        0x114468805 - std::panicking::default_hook::{{closure}}::h552de0233eed7dab
   4:        0x1144683ef - std::panicking::default_hook::hf4e8e1e5a5c43b90
   5:        0x114468ef0 - std::panicking::rust_panic_with_hook::h7c7e0153f3e14d6b
   6:        0x11446898e - std::panicking::begin_panic_handler::{{closure}}::h3c7f7ffd2b05c635
   7:        0x114467a47 - std::sys_common::backtrace::__rust_end_short_backtrace::h3bdf4f6c89eee6ea
   8:        0x1144688fa - _rust_begin_unwind
   9:        0x114489d6f - core::panicking::panic_fmt::h88a1b6fbb9084d2c
  10:        0x114489e55 - core::result::unwrap_failed::h8abe20a396a5e82a
  11:        0x114403a35 - _toggle_key
  12:        0x1100571e2 - _ffi_call_unix64
  13:        0x110056c35 - _ffi_call_int
  14:        0x10f6eff40 - deno_ffi::ffi_call::h7b8d00a17ac9084e
  15:        0x10f9613b3 - deno_core::ops_json::op_sync::{{closure}}::h2dff410c79672602
  16:        0x10f67b81f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd8d69a3a2de8f014
  17:        0x10f67074c - deno_core::ops::OpTable::route_op::h86ef2e6087baaab6
  18:        0x10f682a8c - <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn::hfa3b6c5b5cdbaae7
  19:        0x1101345d5 - __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
  20:        0x110133bc1 - __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
  21:        0x11013312e - __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
fatal runtime error: failed to initiate panic, error 5
@nnmrts nnmrts changed the title Modifier keys don't really seem to work. Modifier keys don't really seem to work Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant