From 1646c0f26cfa21a7023d404008e4d0aa4917193d Mon Sep 17 00:00:00 2001 From: James Young <18669334+noroadsleft@users.noreply.github.com> Date: Sat, 29 May 2021 14:38:50 -0700 Subject: 2021 May 29 Breaking Changes Update (#13034) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add Per Key functionality for AutoShift (#11536) * LED Matrix: Reactive effect buffers & advanced indicators (#12588) * [Keyboard] kint36: switch to sym_eager_pk debouncing (#12626) * [Keyboard] kint2pp: reduce input latency by ≈10ms (#12625) * LED Matrix: Split (#12633) * [CI] Format code according to conventions (#12650) * feat: infinite timeout for leader key (#6580) * feat: implement leader_no_timeout logic * docs(leader_key): infinite leader timeout docs * Format code according to conventions (#12680) * Update ADC driver for STM32F1xx, STM32F3xx, STM32F4xx (#12403) * Fix default ADC_RESOLUTION for ADCv3 (and ADCv4) Recent ChibiOS update removed ADC_CFGR1_RES_10BIT from the ADCv3 headers (that macro should not have been there, because ADCv3 has CFGR instead of CFGR1). Fix the default value for ADC_RESOLUTION to use ADC_CFGR_RES_10BITS if it is defined (that name is used for ADCv3 and ADCv4). * Update ADC docs to match the actually used resolution ADC driver for ChibiOS actually uses the 10-bit resolution by default (probably to match AVR); fix the documentation accordingly. Also add both ADC_CFGR_RES_10BITS and ADC_CFGR1_RES_10BIT constants (these names differ according to the ADC implementation in the particular MCU). * Fix pinToMux() for B12 and B13 on STM32F3xx Testing on STM32F303CCT6 revealed that the ADC mux values for B12 and B13 pins were wrong. * Add support for all possible analog pins on STM32F1xx Added ADC mux values for pins A0...A7, B0, B1, C0...C5 on STM32F1xx (they are the same at least for STM32F103x8 and larger F103 devices, and also F102, F105, F107 families). Actually tested on STM32F103C8T6 (therefore pins C0...C5 were not tested). Pins F6...F10, which are present on STM32F103x[C-G] in 144-pin packages, cannot be supported at the moment, because those pins are connected only to ADC3, but the ChibiOS ADC driver for STM32F1xx supports only ADC1. * Add support for all possible analog pins on STM32F4xx Added ADC mux values for pins A0...A7, B0, B1, C0...C5 and optionally F3...F10 (if STM32_ADC_USE_ADC3 is enabled). These mux values are apparently the same for all F4xx devices, except some smaller devices may not have ADC3. Actually tested on STM32F401CCU6, STM32F401CEU6, STM32F411CEU6 (using various WeAct “Blackpill” boards); only pins A0...A7, B0, B1 were tested. Pins F3...F10 are inside `#if STM32_ADC_USE_ADC3` because some devices which don't have ADC3 also don't have the GPIOF port, therefore the code which refers to Fx pins does not compile. * Fix STM32F3xx ADC mux table in documentation The ADC driver documentation had some errors in the mux table for STM32F3xx. Fix this table to match the datasheet and the actual code (mux settings for B12 and B13 were also tested on a real STM32F303CCT6 chip). * Add STM32F1xx ADC pins to the documentation * Add STM32F4xx ADC pins to the documentation * Add initial support for tinyuf2 bootloader (when hosted on F411 blackpill) (#12600) * Add support for jumping to tinyuf2 bootloader. Adds blackpill UF2 example. * Update flashing.md * Update chconf.h * Update config.h * Update halconf.h * Update mcuconf.h * eeprom driver: Refactor where eeprom driver initialisation (and EEPROM emulation initialisation) occurs to make it non-target-specific. (#12671) * Add support for MCU = STM32F446 (#12619) * Add support for MCU = STM32F446 * Update platforms/chibios/GENERIC_STM32_F446XE/configs/config.h * Restore mcuconf.h to the one used by RT-STM32F446RE-NUCLEO64 * stm32f446: update mcuconf.h and board.h for 16MHz operation, with USB enabled, and other peripherals disabled. * Format code according to conventions (#12682) * Format code according to conventions (#12687) * Add STM32L433 and L443 support (#12063) * initial L433 commit * change to XC * fix L433 * disable all peripherals * update system and peripheral clocks * 433 change * use its own board files * revert its own board files * l433 specific change * fix stm32l432xx define * remove duplicate #define * fix bootloader jump * move to L443xx and add i2c2, spi2, usart3 to mcuconf.h * move to L443 * move to L443 * fix sdmmc in mcuconf.h * include STM32L443 * add L443 * Include L443 in compatible microcontrollers * Include L443 in compatible microcontrollers * Update config bootloader jump description * Update ChibiOS define reasoning * Update quantum/mcu_selection.mk * fix git conflict * Updated Function96 with V2 files and removed chconf.h and halconf.h (#12613) * Fix bad PR merge for #6580. (#12721) * Change RGB/LED Matrix to use a simple define for USB suspend (#12697) * [CI] Format code according to conventions (#12731) * Fixing transport's led/rgb matrix suspend state logic (#12770) * [CI] Format code according to conventions (#12772) * Fix comment parsing (#12750) * Added OLED fade out support (#12086) * fix some references to bin/qmk that slipped in (#12832) * Resolve a number of warnings in `qmk generate-api` (#12833) * New command: qmk console (#12828) * stash poc * stash * tidy up implementation * Tidy up slightly for review * Tidy up slightly for review * Bodge environment to make tests pass * Refactor away from asyncio due to windows issues * Filter devices * align vid/pid printing * Add hidapi to the installers * start preparing for multiple hid_listeners * udev rules for hid_listen * refactor to move closer to end state * very basic implementation of the threaded model * refactor how vid/pid/index are supplied and parsed * windows improvements * read the report directly when usage page isn't available * add per-device colors, the choice to show names or numbers, and refactor * add timestamps * Add support for showing bootloaders * tweak the color for bootloaders * Align bootloader disconnect with connect color * add support for showing all bootloaders * fix the pyusb check * tweaks * fix exception * hide a stack trace behind -v * add --no-bootloaders option * add documentation for qmk console * Apply suggestions from code review * pyformat * clean up and flesh out KNOWN_BOOTLOADERS * Remove pointless SERIAL_LINK_ENABLE rules (#12846) * Make Swap Hands use PROGMEM (#12284) This converts the array that the Swap Hands feature uses to use PROGMEM, and to read from that array, as such. Since this array never changes at runtime, there is no reason to keep it in memory. Especially for AVR boards, as memory is a precious resource. * Fix another bin/qmk reference (#12856) * [Keymap] Turn OLED off on suspend in soundmonster keymap (#10419) * Fixup build errors on `develop` branch. (#12723) * LED Matrix: Effects! (#12651) * Fix syntax error when compiling for ARM (#12866) * Remove KEYMAP and LAYOUT_kc (#12160) * alias KEYMAP to LAYOUT * remove KEYMAP and LAYOUT_kc * Add setup, clone, and env to the list of commands we allow even with broken modules (#12868) * Rename `point_t` -> `led_point_t` (#12864) * [Keyboard] updated a vendor name / fixed minor keymap issues (#12881) * Add missing LED Matrix suspend code to suspend.c (#12878) * LED Matrix: Documentation (#12685) * Deprecate `send_unicode_hex_string()` (#12602) * Fix spelling mistake regarding LED Matrix in split_common. (#12888) * [Keymap] Fix QWERTY/DVORAK status output for kzar keymap (#12895) * Use milc.subcommand.config instead of qmk.cli.config (#12915) * Use milc.subcommand.config instead * pyformat * remove the config test * Add function to allow repeated blinking of one layer (#12237) * Implement function rgblight_blink_layer_repeat to allow repeated blinking of one layer at a time * Update doc * Rework rgblight blinking according to requested change * optimize storage * Fixup housekeeping from being invoked twice per loop. (#12933) * matrix: wait for row signal to go HIGH for every row (#12945) I noticed this discrepancy (last row of the matrix treated differently than the others) when optimizing the input latency of my keyboard controller, see also https://michael.stapelberg.ch/posts/2021-05-08-keyboard-input-latency-qmk-kinesis/ Before this commit, when tuning the delays I noticed ghost key presses when pressing the F2 key, which is on the last row of the keyboard matrix: the dead_grave key, which is on the first row of the keyboard matrix, would be incorrectly detected as pressed. After this commit, all keyboard matrix rows are interpreted correctly. I suspect that my setup is more susceptible to this nuance than others because I use GPIO_INPUT_PIN_DELAY=0 and hence don’t have another delay that might mask the problem. * ensure we do not conflict with existing keymap aliases (#12976) * Add support for up to 4 IS31FL3733 drivers (#12342) * Convert Encoder callbacks to be boolean functions (#12805) * [Keyboard] Fix Terrazzo build failure (#12977) * Do not hard set config in CPTC files (#11864) * [Keyboard] Corne - Remove legacy revision support (#12226) * [Keymap] Update to Drashna keymap and user code (based on develop) (#12936) * Add Full-duplex serial driver for ARM boards (#9842) * Document LED_MATRIX_FRAMEBUFFER_EFFECTS (#12987) * Backlight: add defines for default level and breathing state (#12560) * Add dire message about LUFA mass storage bootloader (#13014) * [Keyboard] Remove redundant legacy and common headers for crkbd (#13023) Was causing compiler errors on some systems. * Fix keyboards/keymaps for boolean encoder callback changes (#12985) * `backlight.c`: include `eeprom.h` (#13024) * Add changelog for 2021-05-29 Breaking Changes merge (#12939) * Add ChangeLog for 2021-05-29 Breaking Changes Merge: initial version * Add recent develop changes * Sort recent develop changes * Remove sections for ChibiOS changes per tzarc No ChibiOS changes this round. * Add and sort recent develop changes * add notes about keyboard moves/deletions * import changelog for PR 12172 Documents the change to BOOTMAGIC_ENABLE. * update section headings * re-sort changelog * add additional note regarding Bootmagic changes * remove changelog timestamp * update dates in main Breaking Changes docs * fix broken section anchors in previous changelogs * add link to backlight/eeprom patch to changelog * highlight some more changes * link PRs from section headers * Restore standard readme * run: qmk cformat --core-only --- lib/python/qmk/c_parse.py | 2 +- lib/python/qmk/cli/__init__.py | 70 ++++--- lib/python/qmk/cli/console.py | 302 ++++++++++++++++++++++++++++++ lib/python/qmk/cli/generate/layouts.py | 4 +- lib/python/qmk/commands.py | 1 + lib/python/qmk/constants.py | 2 +- lib/python/qmk/tests/test_cli_commands.py | 4 +- 7 files changed, 352 insertions(+), 33 deletions(-) create mode 100644 lib/python/qmk/cli/console.py (limited to 'lib/python/qmk') diff --git a/lib/python/qmk/c_parse.py b/lib/python/qmk/c_parse.py index d4f39c8839..991373d569 100644 --- a/lib/python/qmk/c_parse.py +++ b/lib/python/qmk/c_parse.py @@ -8,7 +8,7 @@ from milc import cli from qmk.comment_remover import comment_remover default_key_entry = {'x': -1, 'y': 0, 'w': 1} -single_comment_regex = re.compile(r' */[/*].*$') +single_comment_regex = re.compile(r'\s+/[/*].*$') multi_comment_regex = re.compile(r'/\*(.|\n)*?\*/', re.MULTILINE) diff --git a/lib/python/qmk/cli/__init__.py b/lib/python/qmk/cli/__init__.py index d07790d118..48812ae4ba 100644 --- a/lib/python/qmk/cli/__init__.py +++ b/lib/python/qmk/cli/__init__.py @@ -12,6 +12,20 @@ from subprocess import run from milc import cli, __VERSION__ from milc.questions import yesno +import_names = { + # A mapping of package name to importable name + 'pep8-naming': 'pep8ext_naming', + 'pyusb': 'usb.core', +} + +safe_commands = [ + # A list of subcommands we always run, even when the module imports fail + 'clone', + 'config', + 'env', + 'setup', +] + def _run_cmd(*command): """Run a command in a subshell. @@ -50,8 +64,8 @@ def _find_broken_requirements(requirements): module_import = module_name.replace('-', '_') # Not every module is importable by its own name. - if module_name == "pep8-naming": - module_import = "pep8ext_naming" + if module_name in import_names: + module_import = import_names[module_name] if not find_spec(module_import): broken_modules.append(module_name) @@ -107,32 +121,31 @@ if int(milc_version[0]) < 2 and int(milc_version[1]) < 3: # Check to make sure we have all our dependencies msg_install = 'Please run `python3 -m pip install -r %s` to install required python dependencies.' - -if _broken_module_imports('requirements.txt'): - if yesno('Would you like to install the required Python modules?'): - _run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt') - else: - print() - print(msg_install % (str(Path('requirements.txt').resolve()),)) - print() - exit(1) - -if cli.config.user.developer: - args = sys.argv[1:] - while args and args[0][0] == '-': - del args[0] - if not args or args[0] != 'config': - if _broken_module_imports('requirements-dev.txt'): - if yesno('Would you like to install the required developer Python modules?'): - _run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements-dev.txt') - elif yesno('Would you like to disable developer mode?'): - _run_cmd(sys.argv[0], 'config', 'user.developer=None') - else: - print() - print(msg_install % (str(Path('requirements-dev.txt').resolve()),)) - print('You can also turn off developer mode: qmk config user.developer=None') - print() - exit(1) +args = sys.argv[1:] +while args and args[0][0] == '-': + del args[0] + +if not args or args[0] not in safe_commands: + if _broken_module_imports('requirements.txt'): + if yesno('Would you like to install the required Python modules?'): + _run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt') + else: + print() + print(msg_install % (str(Path('requirements.txt').resolve()),)) + print() + exit(1) + + if cli.config.user.developer and _broken_module_imports('requirements-dev.txt'): + if yesno('Would you like to install the required developer Python modules?'): + _run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements-dev.txt') + elif yesno('Would you like to disable developer mode?'): + _run_cmd(sys.argv[0], 'config', 'user.developer=None') + else: + print() + print(msg_install % (str(Path('requirements-dev.txt').resolve()),)) + print('You can also turn off developer mode: qmk config user.developer=None') + print() + exit(1) # Import our subcommands from . import bux # noqa @@ -142,6 +155,7 @@ from . import chibios # noqa from . import clean # noqa from . import compile # noqa from milc.subcommand import config # noqa +from . import console # noqa from . import docs # noqa from . import doctor # noqa from . import fileformat # noqa diff --git a/lib/python/qmk/cli/console.py b/lib/python/qmk/cli/console.py new file mode 100644 index 0000000000..45ff0c8bee --- /dev/null +++ b/lib/python/qmk/cli/console.py @@ -0,0 +1,302 @@ +"""Acquire debugging information from usb hid devices + +cli implementation of https://www.pjrc.com/teensy/hid_listen.html +""" +from pathlib import Path +from threading import Thread +from time import sleep, strftime + +import hid +import usb.core + +from milc import cli + +LOG_COLOR = { + 'next': 0, + 'colors': [ + '{fg_blue}', + '{fg_cyan}', + '{fg_green}', + '{fg_magenta}', + '{fg_red}', + '{fg_yellow}', + ], +} + +KNOWN_BOOTLOADERS = { + # VID , PID + ('03EB', '2FEF'): 'atmel-dfu: ATmega16U2', + ('03EB', '2FF0'): 'atmel-dfu: ATmega32U2', + ('03EB', '2FF3'): 'atmel-dfu: ATmega16U4', + ('03EB', '2FF4'): 'atmel-dfu: ATmega32U4', + ('03EB', '2FF9'): 'atmel-dfu: AT90USB64', + ('03EB', '2FFA'): 'atmel-dfu: AT90USB162', + ('03EB', '2FFB'): 'atmel-dfu: AT90USB128', + ('03EB', '6124'): 'Microchip SAM-BA', + ('0483', 'DF11'): 'stm32-dfu: STM32 BOOTLOADER', + ('16C0', '05DC'): 'USBasp: USBaspLoader', + ('16C0', '05DF'): 'bootloadHID: HIDBoot', + ('16C0', '0478'): 'halfkay: Teensy Halfkay', + ('1B4F', '9203'): 'caterina: Pro Micro 3.3V', + ('1B4F', '9205'): 'caterina: Pro Micro 5V', + ('1B4F', '9207'): 'caterina: LilyPadUSB', + ('1C11', 'B007'): 'kiibohd: Kiibohd DFU Bootloader', + ('1EAF', '0003'): 'stm32duino: Maple 003', + ('1FFB', '0101'): 'caterina: Polou A-Star 32U4 Bootloader', + ('2341', '0036'): 'caterina: Arduino Leonardo', + ('2341', '0037'): 'caterina: Arduino Micro', + ('239A', '000C'): 'caterina: Adafruit Feather 32U4', + ('239A', '000D'): 'caterina: Adafruit ItsyBitsy 32U4 3v', + ('239A', '000E'): 'caterina: Adafruit ItsyBitsy 32U4 5v', + ('239A', '000E'): 'caterina: Adafruit ItsyBitsy 32U4 5v', + ('2A03', '0036'): 'caterina: Arduino Leonardo', + ('2A03', '0037'): 'caterina: Arduino Micro', + ('314B', '0106'): 'apm32-dfu: APM32 DFU ISP Mode' +} + + +class MonitorDevice(object): + def __init__(self, hid_device, numeric): + self.hid_device = hid_device + self.numeric = numeric + self.device = hid.Device(path=hid_device['path']) + self.current_line = '' + + cli.log.info('Console Connected: %(color)s%(manufacturer_string)s %(product_string)s{style_reset_all} (%(color)s%(vendor_id)04X:%(product_id)04X:%(index)d{style_reset_all})', hid_device) + + def read(self, size, encoding='ascii', timeout=1): + """Read size bytes from the device. + """ + return self.device.read(size, timeout).decode(encoding) + + def read_line(self): + """Read from the device's console until we get a \n. + """ + while '\n' not in self.current_line: + self.current_line += self.read(32).replace('\x00', '') + + lines = self.current_line.split('\n', 1) + self.current_line = lines[1] + + return lines[0] + + def run_forever(self): + while True: + try: + message = {**self.hid_device, 'text': self.read_line()} + identifier = (int2hex(message['vendor_id']), int2hex(message['product_id'])) if self.numeric else (message['manufacturer_string'], message['product_string']) + message['identifier'] = ':'.join(identifier) + message['ts'] = '{style_dim}{fg_green}%s{style_reset_all} ' % (strftime(cli.config.general.datetime_fmt),) if cli.args.timestamp else '' + + cli.echo('%(ts)s%(color)s%(identifier)s:%(index)d{style_reset_all}: %(text)s' % message) + + except hid.HIDException: + break + + +class FindDevices(object): + def __init__(self, vid, pid, index, numeric): + self.vid = vid + self.pid = pid + self.index = index + self.numeric = numeric + + def run_forever(self): + """Process messages from our queue in a loop. + """ + live_devices = {} + live_bootloaders = {} + + while True: + try: + for device in list(live_devices): + if not live_devices[device]['thread'].is_alive(): + cli.log.info('Console Disconnected: %(color)s%(manufacturer_string)s %(product_string)s{style_reset_all} (%(color)s%(vendor_id)04X:%(product_id)04X:%(index)d{style_reset_all})', live_devices[device]) + del live_devices[device] + + for device in self.find_devices(): + if device['path'] not in live_devices: + device['color'] = LOG_COLOR['colors'][LOG_COLOR['next']] + LOG_COLOR['next'] = (LOG_COLOR['next'] + 1) % len(LOG_COLOR['colors']) + live_devices[device['path']] = device + + try: + monitor = MonitorDevice(device, self.numeric) + device['thread'] = Thread(target=monitor.run_forever, daemon=True) + + device['thread'].start() + except Exception as e: + device['e'] = e + device['e_name'] = e.__class__.__name__ + cli.log.error("Could not connect to %(color)s%(manufacturer_string)s %(product_string)s{style_reset_all} (%(color)s:%(vendor_id)04X:%(product_id)04X:%(index)d): %(e_name)s: %(e)s", device) + if cli.config.general.verbose: + cli.log.exception(e) + del live_devices[device['path']] + + if cli.args.bootloaders: + for device in self.find_bootloaders(): + if device.address in live_bootloaders: + live_bootloaders[device.address]._qmk_found = True + else: + name = KNOWN_BOOTLOADERS[(int2hex(device.idVendor), int2hex(device.idProduct))] + cli.log.info('Bootloader Connected: {style_bright}{fg_magenta}%s', name) + device._qmk_found = True + live_bootloaders[device.address] = device + + for device in list(live_bootloaders): + if live_bootloaders[device]._qmk_found: + live_bootloaders[device]._qmk_found = False + else: + name = KNOWN_BOOTLOADERS[(int2hex(live_bootloaders[device].idVendor), int2hex(live_bootloaders[device].idProduct))] + cli.log.info('Bootloader Disconnected: {style_bright}{fg_magenta}%s', name) + del live_bootloaders[device] + + sleep(.1) + + except KeyboardInterrupt: + break + + def is_bootloader(self, hid_device): + """Returns true if the device in question matches a known bootloader vid/pid. + """ + return (int2hex(hid_device.idVendor), int2hex(hid_device.idProduct)) in KNOWN_BOOTLOADERS + + def is_console_hid(self, hid_device): + """Returns true when the usage page indicates it's a teensy-style console. + """ + return hid_device['usage_page'] == 0xFF31 and hid_device['usage'] == 0x0074 + + def is_filtered_device(self, hid_device): + """Returns True if the device should be included in the list of available consoles. + """ + return int2hex(hid_device['vendor_id']) == self.vid and int2hex(hid_device['product_id']) == self.pid + + def find_devices_by_report(self, hid_devices): + """Returns a list of available teensy-style consoles by doing a brute-force search. + + Some versions of linux don't report usage and usage_page. In that case we fallback to reading the report (possibly inaccurately) ourselves. + """ + devices = [] + + for device in hid_devices: + path = device['path'].decode('utf-8') + + if path.startswith('/dev/hidraw'): + number = path[11:] + report = Path(f'/sys/class/hidraw/hidraw{number}/device/report_descriptor') + + if report.exists(): + rp = report.read_bytes() + + if rp[1] == 0x31 and rp[3] == 0x09: + devices.append(device) + + return devices + + def find_bootloaders(self): + """Returns a list of available bootloader devices. + """ + return list(filter(self.is_bootloader, usb.core.find(find_all=True))) + + def find_devices(self): + """Returns a list of available teensy-style consoles. + """ + hid_devices = hid.enumerate() + devices = list(filter(self.is_console_hid, hid_devices)) + + if not devices: + devices = self.find_devices_by_report(hid_devices) + + if self.vid and self.pid: + devices = list(filter(self.is_filtered_device, devices)) + + # Add index numbers + device_index = {} + for device in devices: + id = ':'.join((int2hex(device['vendor_id']), int2hex(device['product_id']))) + + if id not in device_index: + device_index[id] = 0 + + device_index[id] += 1 + device['index'] = device_index[id] + + return devices + + +def int2hex(number): + """Returns a string representation of the number as hex. + """ + return "%04X" % number + + +def list_devices(device_finder): + """Show the user a nicely formatted list of devices. + """ + devices = device_finder.find_devices() + + if devices: + cli.log.info('Available devices:') + for dev in devices: + color = LOG_COLOR['colors'][LOG_COLOR['next']] + LOG_COLOR['next'] = (LOG_COLOR['next'] + 1) % len(LOG_COLOR['colors']) + cli.log.info("\t%s%s:%s:%d{style_reset_all}\t%s %s", color, int2hex(dev['vendor_id']), int2hex(dev['product_id']), dev['index'], dev['manufacturer_string'], dev['product_string']) + + if cli.args.bootloaders: + bootloaders = device_finder.find_bootloaders() + + if bootloaders: + cli.log.info('Available Bootloaders:') + + for dev in bootloaders: + cli.log.info("\t%s:%s\t%s", int2hex(dev.idVendor), int2hex(dev.idProduct), KNOWN_BOOTLOADERS[(int2hex(dev.idVendor), int2hex(dev.idProduct))]) + + +@cli.argument('--bootloaders', arg_only=True, default=True, action='store_boolean', help='displaying bootloaders.') +@cli.argument('-d', '--device', help='Device to select - uses format :[:].') +@cli.argument('-l', '--list', arg_only=True, action='store_true', help='List available hid_listen devices.') +@cli.argument('-n', '--numeric', arg_only=True, action='store_true', help='Show VID/PID instead of names.') +@cli.argument('-t', '--timestamp', arg_only=True, action='store_true', help='Print the timestamp for received messages as well.') +@cli.argument('-w', '--wait', type=int, default=1, help="How many seconds to wait between checks (Default: 1)") +@cli.subcommand('Acquire debugging information from usb hid devices.', hidden=False if cli.config.user.developer else True) +def console(cli): + """Acquire debugging information from usb hid devices + """ + vid = None + pid = None + index = 1 + + if cli.config.console.device: + device = cli.config.console.device.split(':') + + if len(device) == 2: + vid, pid = device + + elif len(device) == 3: + vid, pid, index = device + + if not index.isdigit(): + cli.log.error('Device index must be a number! Got "%s" instead.', index) + exit(1) + + index = int(index) + + if index < 1: + cli.log.error('Device index must be greater than 0! Got %s', index) + exit(1) + + else: + cli.log.error('Invalid format for device, expected ":[:]" but got "%s".', cli.config.console.device) + cli.print_help() + exit(1) + + vid = vid.upper() + pid = pid.upper() + + device_finder = FindDevices(vid, pid, index, cli.args.numeric) + + if cli.args.list: + return list_devices(device_finder) + + print('Looking for devices...', flush=True) + device_finder.run_forever() diff --git a/lib/python/qmk/cli/generate/layouts.py b/lib/python/qmk/cli/generate/layouts.py index ad6946d6cf..4de982f822 100755 --- a/lib/python/qmk/cli/generate/layouts.py +++ b/lib/python/qmk/cli/generate/layouts.py @@ -85,7 +85,9 @@ def generate_layouts(cli): for alias, target in kb_info_json.get('layout_aliases', {}).items(): layouts_h_lines.append('') - layouts_h_lines.append('#define %s %s' % (alias, target)) + layouts_h_lines.append(f'#ifndef {alias}') + layouts_h_lines.append(f'# define {alias} {target}') + layouts_h_lines.append('#endif') # Show the results layouts_h = '\n'.join(layouts_h_lines) + '\n' diff --git a/lib/python/qmk/commands.py b/lib/python/qmk/commands.py index ee049e8af7..3a35c11031 100644 --- a/lib/python/qmk/commands.py +++ b/lib/python/qmk/commands.py @@ -201,6 +201,7 @@ def compile_configurator_json(user_keymap, bootloader=None, parallel=1, **env_va f'VERBOSE={verbose}', f'COLOR={color}', 'SILENT=false', + f'QMK_BIN={"bin/qmk" if "DEPRECATED_BIN_QMK" in os.environ else "qmk"}', ]) return make_command diff --git a/lib/python/qmk/constants.py b/lib/python/qmk/constants.py index 3ed69f3bf9..49e5e0eb42 100644 --- a/lib/python/qmk/constants.py +++ b/lib/python/qmk/constants.py @@ -10,7 +10,7 @@ QMK_FIRMWARE = Path.cwd() MAX_KEYBOARD_SUBFOLDERS = 5 # Supported processor types -CHIBIOS_PROCESSORS = 'cortex-m0', 'cortex-m0plus', 'cortex-m3', 'cortex-m4', 'MKL26Z64', 'MK20DX128', 'MK20DX256', 'STM32F042', 'STM32F072', 'STM32F103', 'STM32F303', 'STM32F401', 'STM32F411', 'STM32G431', 'STM32G474' +CHIBIOS_PROCESSORS = 'cortex-m0', 'cortex-m0plus', 'cortex-m3', 'cortex-m4', 'MKL26Z64', 'MK20DX128', 'MK20DX256', 'MK66F18', 'STM32F042', 'STM32F072', 'STM32F103', 'STM32F303', 'STM32F401', 'STM32F411', 'STM32F446', 'STM32G431', 'STM32G474', 'STM32L433', 'STM32L443' LUFA_PROCESSORS = 'at90usb162', 'atmega16u2', 'atmega32u2', 'atmega16u4', 'atmega32u4', 'at90usb646', 'at90usb647', 'at90usb1286', 'at90usb1287', None VUSB_PROCESSORS = 'atmega32a', 'atmega328p', 'atmega328', 'attiny85' diff --git a/lib/python/qmk/tests/test_cli_commands.py b/lib/python/qmk/tests/test_cli_commands.py index a7b70a7d99..afdbc81429 100644 --- a/lib/python/qmk/tests/test_cli_commands.py +++ b/lib/python/qmk/tests/test_cli_commands.py @@ -7,7 +7,7 @@ is_windows = 'windows' in platform.platform().lower() def check_subcommand(command, *args): - cmd = ['bin/qmk', command, *args] + cmd = ['qmk', command, *args] result = cli.run(cmd, stdin=DEVNULL, combined_output=True) return result @@ -16,7 +16,7 @@ def check_subcommand_stdin(file_to_read, command, *args): """Pipe content of a file to a command and return output. """ with open(file_to_read, encoding='utf-8') as my_file: - cmd = ['bin/qmk', command, *args] + cmd = ['qmk', command, *args] result = cli.run(cmd, stdin=my_file, combined_output=True) return result -- cgit v1.2.3 From 305cca9a5e4c304b0883f07ab4e5de9a493fdbd9 Mon Sep 17 00:00:00 2001 From: Erovia Date: Sun, 30 May 2021 16:17:40 +0100 Subject: CLI: Fix automagic (#13046) --- lib/python/qmk/decorators.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'lib/python/qmk') diff --git a/lib/python/qmk/decorators.py b/lib/python/qmk/decorators.py index 8d43ae980f..f6270990b9 100644 --- a/lib/python/qmk/decorators.py +++ b/lib/python/qmk/decorators.py @@ -17,12 +17,12 @@ def automagic_keyboard(func): @functools.wraps(func) def wrapper(*args, **kwargs): # Ensure that `--keyboard` was not passed and CWD is under `qmk_firmware/keyboards` - if cli.config_source[cli._entrypoint.__name__]['keyboard'] != 'argument': + if cli.config_source[cli._subcommand.__name__]['keyboard'] != 'argument': keyboard = find_keyboard_from_dir() if keyboard: - cli.config[cli._entrypoint.__name__]['keyboard'] = keyboard - cli.config_source[cli._entrypoint.__name__]['keyboard'] = 'keyboard_directory' + cli.config[cli._subcommand.__name__]['keyboard'] = keyboard + cli.config_source[cli._subcommand.__name__]['keyboard'] = 'keyboard_directory' return func(*args, **kwargs) @@ -37,12 +37,12 @@ def automagic_keymap(func): @functools.wraps(func) def wrapper(*args, **kwargs): # Ensure that `--keymap` was not passed and that we're under `qmk_firmware` - if cli.config_source[cli._entrypoint.__name__]['keymap'] != 'argument': + if cli.config_source[cli._subcommand.__name__]['keymap'] != 'argument': keymap_name, keymap_type = find_keymap_from_dir() if keymap_name: - cli.config[cli._entrypoint.__name__]['keymap'] = keymap_name - cli.config_source[cli._entrypoint.__name__]['keymap'] = keymap_type + cli.config[cli._subcommand.__name__]['keymap'] = keymap_name + cli.config_source[cli._subcommand.__name__]['keymap'] = keymap_type return func(*args, **kwargs) -- cgit v1.2.3 From af91221ddbf35199495626da44847737588eabe9 Mon Sep 17 00:00:00 2001 From: Ryan Date: Fri, 4 Jun 2021 01:53:03 +1000 Subject: `qmk multibuild`: Fix bug with Path objects (#13093) --- lib/python/qmk/cli/multibuild.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'lib/python/qmk') diff --git a/lib/python/qmk/cli/multibuild.py b/lib/python/qmk/cli/multibuild.py index 46594c0997..2848a3b829 100755 --- a/lib/python/qmk/cli/multibuild.py +++ b/lib/python/qmk/cli/multibuild.py @@ -76,4 +76,4 @@ all: {keyboard_safe}_binary ) # yapf: enable - cli.run([make_cmd, '-j', str(cli.args.parallel), '-f', makefile, 'all'], capture_output=False, stdin=DEVNULL) + cli.run([make_cmd, '-j', str(cli.args.parallel), '-f', makefile.as_posix(), 'all'], capture_output=False, stdin=DEVNULL) -- cgit v1.2.3 From 21aa7890cbff6368cf75c84a083f022141e0e4ca Mon Sep 17 00:00:00 2001 From: Nick Brassel Date: Wed, 9 Jun 2021 17:25:21 +1000 Subject: Allow limiting the multibuild based on keymap name. (#13066) --- lib/python/qmk/cli/multibuild.py | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) (limited to 'lib/python/qmk') diff --git a/lib/python/qmk/cli/multibuild.py b/lib/python/qmk/cli/multibuild.py index 2848a3b829..bdb0b493c8 100755 --- a/lib/python/qmk/cli/multibuild.py +++ b/lib/python/qmk/cli/multibuild.py @@ -2,6 +2,7 @@ This will compile everything in parallel, for testing purposes. """ +import os import re from pathlib import Path from subprocess import DEVNULL @@ -11,6 +12,7 @@ from milc import cli from qmk.constants import QMK_FIRMWARE from qmk.commands import _find_make import qmk.keyboard +import qmk.keymap def _make_rules_mk_filter(key, value): @@ -29,6 +31,7 @@ def _is_split(keyboard_name): @cli.argument('-j', '--parallel', type=int, default=1, help="Set the number of parallel make jobs to run.") @cli.argument('-c', '--clean', arg_only=True, action='store_true', help="Remove object files before compiling.") @cli.argument('-f', '--filter', arg_only=True, action='append', default=[], help="Filter the list of keyboards based on the supplied value in rules.mk. Supported format is 'SPLIT_KEYBOARD=yes'. May be passed multiple times.") +@cli.argument('-km', '--keymap', type=str, default='default', help="The keymap name to build. Default is 'default'.") @cli.subcommand('Compile QMK Firmware for all keyboards.', hidden=False if cli.config.user.developer else True) def multibuild(cli): """Compile QMK Firmware against all keyboards. @@ -57,23 +60,29 @@ def multibuild(cli): builddir.mkdir(parents=True, exist_ok=True) with open(makefile, "w") as f: for keyboard_name in keyboard_list: - keyboard_safe = keyboard_name.replace('/', '_') - # yapf: disable - f.write( - f"""\ + if qmk.keymap.locate_keymap(keyboard_name, cli.args.keymap) is not None: + keyboard_safe = keyboard_name.replace('/', '_') + # yapf: disable + f.write( + f"""\ all: {keyboard_safe}_binary {keyboard_safe}_binary: @rm -f "{QMK_FIRMWARE}/.build/failed.log.{keyboard_safe}" || true - +@$(MAKE) -C "{QMK_FIRMWARE}" -f "{QMK_FIRMWARE}/build_keyboard.mk" KEYBOARD="{keyboard_name}" KEYMAP="default" REQUIRE_PLATFORM_KEY= COLOR=true SILENT=false \\ - >>"{QMK_FIRMWARE}/.build/build.log.{keyboard_safe}" 2>&1 \\ - || cp "{QMK_FIRMWARE}/.build/build.log.{keyboard_safe}" "{QMK_FIRMWARE}/.build/failed.log.{keyboard_safe}" - @{{ grep '\[ERRORS\]' "{QMK_FIRMWARE}/.build/build.log.{keyboard_safe}" >/dev/null 2>&1 && printf "Build %-64s \e[1;31m[ERRORS]\e[0m\\n" "{keyboard_name}:default" ; }} \\ - || {{ grep '\[WARNINGS\]' "{QMK_FIRMWARE}/.build/build.log.{keyboard_safe}" >/dev/null 2>&1 && printf "Build %-64s \e[1;33m[WARNINGS]\e[0m\\n" "{keyboard_name}:default" ; }} \\ - || printf "Build %-64s \e[1;32m[OK]\e[0m\\n" "{keyboard_name}:default" - @rm -f "{QMK_FIRMWARE}/.build/build.log.{keyboard_safe}" || true + +@$(MAKE) -C "{QMK_FIRMWARE}" -f "{QMK_FIRMWARE}/build_keyboard.mk" KEYBOARD="{keyboard_name}" KEYMAP="{cli.args.keymap}" REQUIRE_PLATFORM_KEY= COLOR=true SILENT=false \\ + >>"{QMK_FIRMWARE}/.build/build.log.{os.getpid()}.{keyboard_safe}" 2>&1 \\ + || cp "{QMK_FIRMWARE}/.build/build.log.{os.getpid()}.{keyboard_safe}" "{QMK_FIRMWARE}/.build/failed.log.{os.getpid()}.{keyboard_safe}" + @{{ grep '\[ERRORS\]' "{QMK_FIRMWARE}/.build/build.log.{os.getpid()}.{keyboard_safe}" >/dev/null 2>&1 && printf "Build %-64s \e[1;31m[ERRORS]\e[0m\\n" "{keyboard_name}:{cli.args.keymap}" ; }} \\ + || {{ grep '\[WARNINGS\]' "{QMK_FIRMWARE}/.build/build.log.{os.getpid()}.{keyboard_safe}" >/dev/null 2>&1 && printf "Build %-64s \e[1;33m[WARNINGS]\e[0m\\n" "{keyboard_name}:{cli.args.keymap}" ; }} \\ + || printf "Build %-64s \e[1;32m[OK]\e[0m\\n" "{keyboard_name}:{cli.args.keymap}" + @rm -f "{QMK_FIRMWARE}/.build/build.log.{os.getpid()}.{keyboard_safe}" || true """# noqa - ) - # yapf: enable + ) + # yapf: enable cli.run([make_cmd, '-j', str(cli.args.parallel), '-f', makefile.as_posix(), 'all'], capture_output=False, stdin=DEVNULL) + + # Check for failures + failures = [f for f in builddir.glob(f'failed.log.{os.getpid()}.*')] + if len(failures) > 0: + return False -- cgit v1.2.3 From 0713797c58376e8237174f8ca83a3fff9d964b9d Mon Sep 17 00:00:00 2001 From: Zach White Date: Mon, 14 Jun 2021 16:51:58 -0700 Subject: Ensure that safe_commands always run (#13199) * ensure that safe_commands always run * import the config subcommand in bin/qmk --- lib/python/qmk/cli/__init__.py | 75 +++++++++++++++++++++------------ lib/python/qmk/cli/chibios/__init__.py | 1 - lib/python/qmk/cli/format/__init__.py | 1 - lib/python/qmk/cli/generate/__init__.py | 9 ---- lib/python/qmk/cli/list/__init__.py | 2 - lib/python/qmk/cli/new/__init__.py | 2 - 6 files changed, 49 insertions(+), 41 deletions(-) (limited to 'lib/python/qmk') diff --git a/lib/python/qmk/cli/__init__.py b/lib/python/qmk/cli/__init__.py index 48812ae4ba..32da1a9b52 100644 --- a/lib/python/qmk/cli/__init__.py +++ b/lib/python/qmk/cli/__init__.py @@ -26,6 +26,42 @@ safe_commands = [ 'setup', ] +subcommands = [ + 'qmk.cli.bux', + 'qmk.cli.c2json', + 'qmk.cli.cformat', + 'qmk.cli.chibios.confmigrate', + 'qmk.cli.clean', + 'qmk.cli.compile', + 'qmk.cli.console', + 'qmk.cli.docs', + 'qmk.cli.doctor', + 'qmk.cli.fileformat', + 'qmk.cli.flash', + 'qmk.cli.format.json', + 'qmk.cli.generate.api', + 'qmk.cli.generate.config_h', + 'qmk.cli.generate.dfu_header', + 'qmk.cli.generate.docs', + 'qmk.cli.generate.info_json', + 'qmk.cli.generate.keyboard_h', + 'qmk.cli.generate.layouts', + 'qmk.cli.generate.rgb_breathe_table', + 'qmk.cli.generate.rules_mk', + 'qmk.cli.hello', + 'qmk.cli.info', + 'qmk.cli.json2c', + 'qmk.cli.lint', + 'qmk.cli.list.keyboards', + 'qmk.cli.list.keymaps', + 'qmk.cli.kle2json', + 'qmk.cli.multibuild', + 'qmk.cli.new.keyboard', + 'qmk.cli.new.keymap', + 'qmk.cli.pyformat', + 'qmk.cli.pytest', +] + def _run_cmd(*command): """Run a command in a subshell. @@ -113,7 +149,7 @@ if sys.version_info[0] != 3 or sys.version_info[1] < 7: milc_version = __VERSION__.split('.') -if int(milc_version[0]) < 2 and int(milc_version[1]) < 3: +if int(milc_version[0]) < 2 and int(milc_version[1]) < 4: requirements = Path('requirements.txt').resolve() print(f'Your MILC library is too old! Please upgrade: python3 -m pip install -U -r {str(requirements)}') @@ -125,7 +161,9 @@ args = sys.argv[1:] while args and args[0][0] == '-': del args[0] -if not args or args[0] not in safe_commands: +safe_command = args and args[0] in safe_commands + +if not safe_command: if _broken_module_imports('requirements.txt'): if yesno('Would you like to install the required Python modules?'): _run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt') @@ -148,27 +186,12 @@ if not args or args[0] not in safe_commands: exit(1) # Import our subcommands -from . import bux # noqa -from . import c2json # noqa -from . import cformat # noqa -from . import chibios # noqa -from . import clean # noqa -from . import compile # noqa -from milc.subcommand import config # noqa -from . import console # noqa -from . import docs # noqa -from . import doctor # noqa -from . import fileformat # noqa -from . import flash # noqa -from . import format # noqa -from . import generate # noqa -from . import hello # noqa -from . import info # noqa -from . import json2c # noqa -from . import lint # noqa -from . import list # noqa -from . import kle2json # noqa -from . import multibuild # noqa -from . import new # noqa -from . import pyformat # noqa -from . import pytest # noqa +for subcommand in subcommands: + try: + __import__(subcommand) + + except ModuleNotFoundError as e: + if safe_command: + print(f'Warning: Could not import {subcommand}: {e.__class__.__name__}, {e}') + else: + raise diff --git a/lib/python/qmk/cli/chibios/__init__.py b/lib/python/qmk/cli/chibios/__init__.py index 4301837def..e69de29bb2 100644 --- a/lib/python/qmk/cli/chibios/__init__.py +++ b/lib/python/qmk/cli/chibios/__init__.py @@ -1 +0,0 @@ -from . import confmigrate diff --git a/lib/python/qmk/cli/format/__init__.py b/lib/python/qmk/cli/format/__init__.py index 741ec778b1..e69de29bb2 100644 --- a/lib/python/qmk/cli/format/__init__.py +++ b/lib/python/qmk/cli/format/__init__.py @@ -1 +0,0 @@ -from . import json diff --git a/lib/python/qmk/cli/generate/__init__.py b/lib/python/qmk/cli/generate/__init__.py index 0efca0022d..e69de29bb2 100644 --- a/lib/python/qmk/cli/generate/__init__.py +++ b/lib/python/qmk/cli/generate/__init__.py @@ -1,9 +0,0 @@ -from . import api -from . import config_h -from . import dfu_header -from . import docs -from . import info_json -from . import keyboard_h -from . import layouts -from . import rgb_breathe_table -from . import rules_mk diff --git a/lib/python/qmk/cli/list/__init__.py b/lib/python/qmk/cli/list/__init__.py index d83cd20b5b..e69de29bb2 100644 --- a/lib/python/qmk/cli/list/__init__.py +++ b/lib/python/qmk/cli/list/__init__.py @@ -1,2 +0,0 @@ -from . import keyboards -from . import keymaps diff --git a/lib/python/qmk/cli/new/__init__.py b/lib/python/qmk/cli/new/__init__.py index fe5d6fe483..e69de29bb2 100644 --- a/lib/python/qmk/cli/new/__init__.py +++ b/lib/python/qmk/cli/new/__init__.py @@ -1,2 +0,0 @@ -from . import keyboard -from . import keymap -- cgit v1.2.3