ぽけこん on ラズベリーパイ導入手順
はじめにPoke-Controller on Raspberry Piを導入にあたって
Poke-Controllerをラズベリーパイで動作させることのメリットとデメリットについて
メリット
動作電力の削減
機器接続環境の簡略化
SDカードの交換による使用用途によってのOSの切り替えが簡単
デメリット
実際に動かす自動化pythonプログラムはWindows環境で作成されている場合が多いため、
動作にあたって入手した自動化ファイルの修正が必要になる可能性がある。
導入にあたってlinuxの操作、ＯSの知識などが必要になる。
本手順ではlinuxについての知識があるものとして説明は省くものとする。
本環境で使用する機器について
RasberryPiOS導入するためのWindowsコンピュータ（teratermのscpにて各種ファイルの転送などを実施）
HDMI付きのテレビ
Nintendo Switch
Raspberry Pi 4B 4GBメモリー版 ￥15,000位 ※現在半導体不足で価格高騰中の為、RaspberryPi400を推奨
ラズベリーパイにつなぐUSBマウス、USBキーボード ￥2,000位 ※RaspberryPi400ならばキーボード不要
MicroSDHCカード 16GB ￥500位
ジャンパーワイヤ ♀♂2本 ￥400位（束で購入可♂♂があるとマイコンのショートに便利、♀♀もあると便利）
ブレッドボード ￥400位
中華キャプチャーボード
MS2109チップのキャプチャーボードでは通常FPS5しか出ない為パッチの適用が必要です
ＭS2109搭載のキャプチャボード ￥800位
MS2130チップ搭載のキャプチャボード ￥2500位
MS2109チップの場合はパッチの適用が必要です
マイコン（pro micro Atmega32U4）￥1800位
raspiOSのダウンロード
OPENCVが64bit対応なため、64bit版のraspiOSをダウンロードします。
2024/09/04 更新
raspios 6.6.31-1+rpt1 (2024-05-29) aarch64 版で更新
2024/09/04時点での手順になりますので、最新のＯSのアップデートにより手順が古くなり同じコマンドなどが実行できなくなっている可能性がありますがご了承ください
raspiOSのインストールツールのダウンロード
Raspberry Pi Imagerを使用し、WindowsOSでraspiOSをインストールします。
https://www.raspberrypi.com/software/
WindowsのスタートメニューからRaspberry Pi Imagerを起動します。
CHOOSE OSボタンを押し、Use customからダウンロードした64bit版の
Raspberry Pi Imagerの64bit版のraspiOSを選択します。
（ダウンロードしたzipファイルのまま選択可能）
CHOOSE STORAGE ボタンを押し、インストール対象のmicroSDHCカードを選択します。
※sdカードは初期化されますので、書き込む場合は必ずsdカードが正しいことを確認してください。
※間違えて大事なファイル等が削除された場合、一切責任を取りませんのでご了承ください。
WRITEボタンを押し、microSDHCカードにRaspiOSをインストールします。
容量にもよりますが、インストールには30分程度かかります。
※完了後、フォーマットしますかとのポップアップが表示されますが、いいえを選択してください。
WindowsからSDカードを取り外し、raspberry piにSDカードを差し込みます。
RasberryPiのHDMIをテレビに接続
RasberryPiのUSBにキーボード、マウス、キャプチャーボードの接続
RasberryPiOSの起動とネットワークの設定の実施
※自分の家の環境に設定してください。
raspiコンフィグにて初期設定
sudo raspi-config
1.3 パスワードの設定 piユーザーのパスワードの設定
1.4 ホスト名の変更 ご希望のホスト名
3.2 sshの有効化 yes
3.6 over serial no
serial port hardware enable yes
Finishにて再起動の実施
config.txtの編集
sudo nano /boot/config.txt
末尾に下記内容を追加してシリアルポートの有効化とBuleToothの停止
[all]
enable_uart=1
dtoverlay=pi3-disable-bt
パッケージリストの更新
sudo apt-get update
ポケコン用Python環境の導入
RasberryPiOSにはインストール後にPythonが導入されていますのでpipコマンドにてパッケージを追加していきます。
2024/09/04更新
OSのPythonからpipで導入できなくなったため、venv環境にて導入します
venvの作成
mkdir ~/pokecon
python3 -m venv pokecon
source ~/pokecon/bin/activate
deactivateコマンド
source ~/pokecon/bin/activate
pip install numpy
pip install opencv-python
pip install pynput
pip install pipenv
deactivateコンソールの再起動
source ~/pokecon/bin/activate
pipenv install pyserial
pip install Pillow
deactivate
sudo apt install python3-pil.imagetk下記パッケージは用途によって導入します。
ポケコンModified verの場合
source ~/pokecon/bin/activate
pip install pygubu
pip install pandas
pip install scipy
deactivateOCRを使用する場合tesseract-OCR
sudo apt install tesseract-ocrsource ~/pokecon/bin/activate
pip install tesseract
pip install pyocr
deactivateLINE Notifyを使用する場合
source ~/pokecon/bin/activate
pip install requests
deactivateポケコンextensionの導入の場合
source ~/pokecon/bin/activate
pip install -r ~/Poke-Controller-Modified-Extension/requirements.txt
pip install numpy
pip install numba
deactivateポケコンの起動
source ~/pokecon/bin/activate
cd ~/Poke-Controller-Modified/SerialController/
python Window.pyポケコンextensionの起動
source ~/pokecon/bin/activate
cd ~/Poke-Controller-Modified-Extension/SerialController/
python Window.pyコマンド
pip install numpy
pip install opencv-python
pip install pynputl
pip install pipenv
exit
コンソールの再起動
pipenv install pyserial
pip install Pillow
sudo apt install python3-pil.imagetk
下記パッケージは用途によって導入します。
ポケコンModified verの場合
pip install pygubu
pip install pandas
pip install scipy
OCRを使用する場合
tesseract-OCR
sudo apt install tesseract-ocr
pip install tesseract
pip install pyocr
pip install pyocr.builder
LINE Notifyを使用する場合
pip install requests
Poke-Controllerのダウンロード ホームディレクトリに解凍
unzip Poke-Controller.zip
本家
https://github.com/KawaSwitch/Poke-Controller
Modified版
https://github.com/Moi-poke/Poke-Controller-Modified
lufaのダウンロード
https://github.com/abcminiuser/lufa
解凍するとlufa-masterフォルダができるので
lufaに名前変更してPoke-Controllerフォルダ配下のlufaフォルダに上書き保存。
ArduinoＩDEのダウンロード ホームディレクトリに解凍
https://www.arduino.cc/en/software
tar xvf arduino-1.8.19-linuxaarch64.tar.xz
マイコンハードウェア情報の編集
nano ~/arduino-1.8.19/hardware/arduino/avr/boards.txt
285 leonardo.vid.1=0x2341
286 leonardo.pid.1=0x8036
↓
285行目：leonardo.vid.1=0x0f0d
286行目：leonardo.pid.1=0x0092
311 leonardo.build.vid=0x2341
312 leonardo.build.pid=0x8036
↓
311行目：leonardo.build.vid=0x0f0d
312行目：leonardo.build.pid=0x0092
LeonardoのMCUに変更
nano ~/Poke-Controller-Modified/makefile
14 MCU = atmega16u2
↓
14 MCU = atmega32u4
Ｌeonardo用hexファイルの作成とマイコンへの書き込み
export PATH=$PATH:/home/pi/arduino-1.8.19/hardware/tools/avr/bin
cd ~/Poke-Controller
make clean
makemakeしたhexファイルをArduino Leonardoに書き込み
ArduinoLeonardoをリセット（RSTとGNDをショートさせる）
以下ここから、以上までをコピー＆ペーストして実行------
ARDUINO_UPLOAD_PORT="$(find /dev/ttyACM* | head -n 1)"
stty -f "${ARDUINO_UPLOAD_PORT}" 1200
while :; do
sleep 0.5
[ -c "${ARDUINO_UPLOAD_PORT}" ] && break
done
avrdude -C /home/pi/arduino-1.8.19/hardware/tools/avr/etc/avrdude.conf -v -patmega32u4 -cavr109 -P"${ARDUINO_UPLOAD_PORT}" -b57600 -D -V -Uflash:w:/home/pi/Poke-Controller/Joystick.hex:i以上---------------------------------------
ラズベリーパイGPIOとマイコンの接続
ジャンパーワイヤーにて接続します。
GND⇔GND
TXD⇒RXD
※今環境はGPIOと直接マイコンを接続するためシリアルアダプタは使用しません
※ＧPIOの接続は間違えた場合最悪基盤が壊れる可能性がありますので正しい場所を結線してください。
※必ず電源を停止した状態で、自己責任の下で行ってください。
ラズベリーパイとマイコンとスイッチの接続図
ラズベリーパイに中華キャプチャーボードを接続したときは特にドライバは必要ありませんでした。(USB2.0のポートに接続しないと認識しませんでした。)
USB VideoのチップがMS2109の場合 プロダクトIDが2109になっている
$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 534d:2109 MacroSilicon USB Video
USB VideoのチップがMS2130の場合 プロダクトIDが2130になっている
$ lsusb
Bus 002 Device 002: ID 345f:2130 UltraSemi USB3 Video
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hubPoke-Controllerのシリアル接続情報の書き換え
USB→AMAに変更
nano ~/Poke-Controller-Modified/SerialController/Commands/Sender.py
print('connecting to ' + "/dev/ttyUSB" + str(portNum))
↓
print('connecting to ' + os.name + "/dev/ttyAMA" + str(portNum))
self.ser = serial.Serial("/dev/ttyUSB" + str(portNum), 9600)
↓
self.ser = serial.Serial("/dev/ttyAMA" + str(portNum), 9600)
Poke-Controllerの起動
source ~/pokecon/bin/activate
cd ~/Poke-Controller-Modified/SerialController/
python Window.py
OPENCVのバージョンの問題か、画像認識時に読み込み失敗エラーが発生するため下記のとおり変更する。
2024/09/04更新
ゆう様の検証により下記の原因が判明したためModified3.0.2.6.2版のみ確認済みですが、Camera.pyとGuiAsset.pyのパッチを適用することによりreadFrameにて失敗することがなくなったため速度の改善が図られました
下記引用
・Poke-Controller-ModifiedのCamera.pyにてcv2のread()関数を実行するだけのスレッドを作り、Lockモジュールを利用したスレッドセーフな画像データの受け渡しをreadFrame()関数にて実現、他スレッドに対して画像データの受け渡しを全てreadFrame()に統一
・MS2109のキャプボはYUYVの圧縮方式だと、ポケコン内部の解像度(1280x720)ではキャプボの画像更新が5～10fpsでしか行われないため、画質は少し落ちるが、圧縮方式をMJPGに変更して60fpsの画像更新を実現（fpsや圧縮方式、バッファサイズの変更するためにVideoCaptureの第二引数にcv2.CAP_V4L2を指定）、キャプボごとにself.camera.setの設定は見直すことを推奨
捕捉
・一応、本パッチの適用してジュナリ様のテラレイド逃げて自動色厳選やフウ様のウッウロボ乱数自動化プログラムがそれなりに動いていることは確認できました。
・圧縮方式をMJPGに変更したので、OCRの解析精度や画像の検出精度が下がる可能性はご留意ください。
・複数スレッドでcv2のread()関数が正常に行われない問題はRaspberry Pi4Bでのみ確認しており、
他OSや他端末で本パッチが有益になるかは未検証(メインの画像描画やコマンドの実行スレッドが少なからず高速化するとは思いますが)です。
・あくまで本パッチはPoke-Controller-Modified用です。
Poke-ControllerやPoke-Controller-Modified-Extensionで適用可能かは未検証です。
PythonCommandBase.pyの編集
nano ~/Poke-Controller-Modified/SerialController/Commands/PythonCommandBase.py
# Read a current image
src = self.camera.readFrame()
src = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY) if use_gray else src
src = src[area[2]:area[3], area[0]:area[1]] if not area else src # trim
# Read a template image
template = cv2.imread(TEMPLATE_PATH+template_path, cv2.IMREAD_GRAYSCALE if use_gray else cv2.IMREAD_COLOR)
template = template[tmp_area[2]:tmp_area[3], tmp_area[0]:tmp_area[1]] # trim
w, h = template.shape[1], template.shape[0]
↓
# Read a current image
while True:
src = self.camera.readFrame()
if src is not None:
break
else:
continue
src = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY) if use_gray else src
src = src[area[2]:area[3], area[0]:area[1]] if area else src # trim
# Read a template image
template = cv2.imread(TEMPLATE_PATH + template_path, cv2.IMREAD_GRAYSCALE if use_gray else cv2.IMREAD_COLOR)
template = template[tmp_area[2]:tmp_area[3], tmp_area[0]:tmp_area[1]] # trim
w, h = template.shape[1], template.shape[0]
↓
# Read a current image
while True:
src = self.camera.readFrame()
if src is not None:
break
else:
continue
src = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY) if use_gray else src
src = src[area[2]:area[3], area[0]:area[1]] if area else src # trim
# Read a template image
template = cv2.imread(TEMPLATE_PATH + template_path, cv2.IMREAD_GRAYSCALE if use_gray else cv2.IMREAD_COLOR)
template = template[tmp_area[2]:tmp_area[3], tmp_area[0]:tmp_area[1]] # trim
w, h = template.shape[1], template.shape[0]
2024/9/4 更新 ゆう様よりいただいたパッチ内容の引用
Modified3.0.2.6.2版のみCamera_thread.patchファイルでの提供
sudo apt install git
cp -p ~/Camera_thread.patch ~/Poke-Controller-Modified/
#現行ファイルのバックアップ
cp -p ~/Poke-Controller-Modified/SerialController/Camera.py ~/Poke-Controller-Modified/SerialController/Camera.py.org
cp -p ~/Poke-Controller-Modified/SerialController/GuiAssets.py ~/Poke-Controller-Modified/SerialController/GuiAssets.py.org
#パッチファイルの移動とファイル確認
cd ~/Poke-Controller-Modified/
ls -l *.patch
#実行結果
-rw-r--r-- 1 pi pi 5712 Sep 3 18:11 Camera_thread.patch
#パッチファイルの適用
patch -p0 --binary < ./Camera_thread.patch
#実行結果
.patchpatching file SerialController/Camera.py
Hunk #2 succeeded at 44 (offset -19 lines).
Hunk #3 succeeded at 77 (offset -19 lines).
Hunk #4 succeeded at 96 (offset -19 lines).
Hunk #5 succeeded at 114 with fuzz 2 (offset -19 lines).
Hunk #6 succeeded at 148 (offset -20 lines).
patching file SerialController/GuiAssets.py下記修正後のCamera.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import cv2
import datetime
from time import sleep
import time
import os
import numpy as np
from logging import getLogger, DEBUG, NullHandler
from threading import Thread, Lock
def imwrite(filename, img, params=None):
#self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
try:
ext = os.path.splitext(filename)[1]
result, n = cv2.imencode(ext, img, params)
if result:
with open(filename, mode='w+b') as f:
n.tofile(f)
return True
else:
return False
except Exception as e:
print(e)
_logger.error(f"Image Write Error: {e}")
return False
class Camera:
def __init__(self, fps=45):
self.camera = None
self.capture_size = (1280, 720)
# self.capture_size = (1920, 1080)
self.capture_dir = "Captures"
self.fps = int(fps)
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
self.buffersize = 4
self.cycle = 0.00
self.started = False
self.read_lock = Lock()
def openCamera(self, cameraId):
if self.camera is not None and self.camera.isOpened():
self._logger.debug("Camera is already opened")
self.destroy()
isV4L2 = False
if os.name == 'nt':
self._logger.debug("NT OS")
self.camera = cv2.VideoCapture(cameraId, cv2.CAP_DSHOW)
# self.camera = cv2.VideoCapture(cameraId)
else:
self._logger.debug("Not NT OS")
##self.camera = cv2.VideoCapture(cameraId)
try:
self.camera = cv2.VideoCapture(cameraId, cv2.CAP_V4L2)
isV4L2 = True
except TypeError:
self.camera = cv2.VideoCapture(cameraId)
if not self.camera.isOpened():
print("Camera ID " + str(cameraId) + " can't open.")
self._logger.error(f"Camera ID {cameraId} cannot open.")
return
print("Camera ID " + str(cameraId) + " opened successfully")
self._logger.debug(f"Camera ID {cameraId} opened successfully.")
# print(self.camera.get(cv2.CAP_PROP_FRAME_WIDTH))
# self.camera.set(cv2.CAP_PROP_FPS, 60)
self.camera.set(cv2.CAP_PROP_FRAME_WIDTH, self.capture_size[0])
self.camera.set(cv2.CAP_PROP_FRAME_HEIGHT, self.capture_size[1])
if isV4L2:
self.camera.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'));
self.camera.set(cv2.CAP_PROP_FPS, 60)
self.camera.set(cv2.CAP_PROP_BUFFERSIZE, self.buffersize)
print("Camera WH:" + str(self.camera.get(cv2.CAP_PROP_FRAME_WIDTH))
+ "x" + str(self.camera.get(cv2.CAP_PROP_FRAME_HEIGHT)))
fourcc_str = int(self.camera.get(cv2.CAP_PROP_FOURCC)).to_bytes(4, 'little').decode('utf-8')
print("Camera Format(After):" + fourcc_str)
print("Camera FPS(After):" + str(self.camera.get(cv2.CAP_PROP_FPS)))
_, self._image_bgr = self.camera.read()
self.camera_read_start()
# self.camera.set(cv2.CAP_PROP_SETTINGS, 0)
def isOpened(self):
self._logger.debug("Camera is opened")
return self.camera.isOpened()
def readFrame(self):
##_, self.image_bgr = self.camera.read()
##return self.image_bgr
self.read_lock.acquire()
frame = self._image_bgr.copy()
self.read_lock.release()
return frame
def saveCapture(self, filename=None, crop=None, crop_ax=None, img=None):
if crop_ax is None:
crop_ax = [0, 0, 1280, 720]
else:
pass
# print(crop_ax)
dt_now = datetime.datetime.now()
if filename is None or filename == "":
filename = dt_now.strftime('%Y-%m-%d_%H-%M-%S') + ".png"
else:
filename = filename + ".png"
image_bgr = self.readFrame()
if crop is None:
##image = self.image_bgr
image = image_bgr
elif crop is 1 or crop is "1":
##image = self.image_bgr[
image = image_bgr[
crop_ax[1]:crop_ax[3],
crop_ax[0]:crop_ax[2]
]
elif crop is 2 or crop is "2":
##image = self.image_bgr[
image = image_bgr[
crop_ax[1]:crop_ax[1] + crop_ax[3],
crop_ax[0]:crop_ax[0] + crop_ax[2]
]
elif img is not None:
image = img
else:
##image = self.image_bgr
image = image_bgr
if not os.path.exists(self.capture_dir):
os.makedirs(self.capture_dir)
self._logger.debug("Created Capture folder")
save_path = os.path.join(self.capture_dir, filename)
try:
imwrite(save_path, image)
self._logger.debug(f"Capture succeeded: {save_path}")
print('capture succeeded: ' + save_path)
except cv2.error as e:
print("Capture Failed")
self._logger.error(f"Capture Failed :{e}")
def destroy(self):
if self.camera is not None and self.camera.isOpened():
self.camera_read_stop()
self.camera.release()
self.camera = None
self._logger.debug("Camera destroyed")
def camera_read_start(self):
if self.started:
print("already started!!")
self.started = True
self.thread = Thread(target=self.camera_update, args=())
self.thread.start()
def camera_read_stop(self) :
self.started = False
self.thread.join()
def camera_update(self) :
while self.started :
_, frame = self.camera.read()
self.read_lock.acquire()
self._image_bgr = frame
self.read_lock.release()
time.sleep(self.cycle)
下記修正後のGuiAsset.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import cv2
import os
import time
import tkinter as tk
from tkinter.scrolledtext import ScrolledText
import numpy as np
import datetime
from collections import deque
from PIL import Image, ImageTk
from Commands import UnitCommand
from Commands import StickCommand
from Commands.Keys import Direction, Stick, Button, Direction, KeyPress
import logging
from logging import INFO, StreamHandler, getLogger, DEBUG, NullHandler
from Commands.PythonCommandBase import PythonCommand, StopThread
try:
os.makedirs('log')
except FileExistsError:
pass
isTakeLog = False
# logger_stick = getLogger(__name__)
nowtime = datetime.datetime.fromtimestamp(time.time()).strftime('%Y%m%d_%H%M%S')
# press button at duration times(s)
class MouseStick(PythonCommand):
NAME = 'MOUSEスティック'
def __init__(self):
super().__init__()
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
def do(self):
pass
def stick(self, buttons, duration=0.1, wait=0.1):
self.keys.input(buttons, ifPrint=False)
self.wait(duration)
self.wait(wait)
# press button at duration times(s)
def stickEnd(self, buttons):
self.keys.inputEnd(buttons)
class CaptureArea(tk.Canvas):
def __init__(self, camera, fps, is_show, ser, master=None, show_width=640, show_height=360):
super().__init__(master, borderwidth=0, cursor='tcross', width=show_width, height=show_height)
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
self.master = master
self.radius = 60 # 描画する円の半径
self.camera = camera
# self.show_size = (640, 360)
self.show_width = int(show_width)
self.show_height = int(show_height)
self.show_size = (self.show_width, self.show_height)
self.is_show_var = is_show
self.lx_init, self.ly_init = 0, 0
self.rx_init, self.ry_init = 0, 0
self.min_x, self.min_y = 0, 0
self.max_x, self.max_y = 0, 0
self.keys = None
self.ser = ser
self.lcircle = None
self.lcircle2 = None
self.rcircle = None
self.rcircle2 = None
self.LStick = None
self.RStick = None
self.calc_time = None
self.ss = None
self.dq = None
self._langle = None
self._lmag = None
self._rangle = None
self._rmag = None
self.stick_handler = StreamHandler()
self.stick_logging_level = DEBUG
self.stick_handler.setLevel(self.stick_logging_level)
# self._logger.setLevel(self.stick_logging_level)
# self._logger.addHandler(self.stick_handler)
# self._logger.propagate = False
if isTakeLog:
filename_base = os.path.join("log", f"{nowtime}")
self.LS = logging.FileHandler(filename=f"{filename_base}_LStick.log", encoding='utf-8')
self.LS.setLevel(logging.DEBUG)
self.LSTICK_logger = logging.getLogger("L_STICK")
self.LSTICK_logger.setLevel(logging.DEBUG)
self.LSTICK_logger.addHandler(self.LS)
self.RS = logging.FileHandler(filename=f"{filename_base}_RStick.log", encoding='utf-8')
self.RS.setLevel(logging.DEBUG)
self.RSTICK_logger = logging.getLogger("R_STICK")
self.RSTICK_logger.setLevel(logging.DEBUG)
self.RSTICK_logger.addHandler(self.RS)
# self.circle =
self.setFps(fps)
self.bind("<Control-ButtonPress-1>", self.mouseCtrlLeftPress)
self.bind("<Control-ButtonRelease-1>", self.mouseCtrlLeftRelease)
self.bind("<Control-Shift-ButtonPress-1>", self.StartRangeSS)
self.bind("<Control-Shift-Button1-Motion>", self.MotionRangeSS)
self.bind("<Control-Shift-ButtonRelease-1>", self.ReleaseRangeSS)
# Set disabled image first
disabled_img = cv2.imread("../Images/disabled.png", cv2.IMREAD_GRAYSCALE)
disabled_pil = Image.fromarray(disabled_img)
self.disabled_tk = ImageTk.PhotoImage(disabled_pil)
self.im = self.disabled_tk
# self.configure(image=self.disabled_tk) # labelからキャンバスに変更したので微修正
self.im_ = self.create_image(0, 0, image=self.disabled_tk, anchor=tk.NW)
def ApplyLStickMouse(self):
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
else:
self.UnbindLeftClick()
def ApplyRStickMouse(self):
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
else:
self.UnbindRightClick()
def StartRangeSS(self, event):
##self.ss = self.camera.image_bgr
image_bgr = self.camera.readFrame()
self.ss = image_bgr
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.UnbindRightClick()
self.min_x, self.min_y = event.x, event.y
self.delete('SelectArea')
self.create_rectangle(self.min_x,
self.min_y,
self.min_x + 1,
self.min_y + 1,
outline='red',
tag='SelectArea')
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(self.min_x, self.min_y,
int(self.min_x * ratio_x),
int(self.min_y * ratio_y)))
self._logger.info('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(self.min_x, self.min_y,
int(self.min_x * ratio_x),
int(self.min_y * ratio_y)))
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
def MotionRangeSS(self, event):
if event.x < 0:
self.max_x = 0
else:
self.max_x = min(self.show_width, event.x)
if event.y < 0:
self.max_y = 0
else:
self.max_y = min(self.show_height, event.y)
self.coords('SelectArea', self.min_x, self.min_y, self.max_x + 1, self.max_y + 1)
self.coords('SelectAreaFilled', self.min_x, self.min_y, self.max_x + 1, self.max_y + 1)
def ReleaseRangeSS(self, event):
# self.max_x, self.max_y = event.x, event.y
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse up: Show ({}, {}) / Capture ({}, {})'.format(self.max_x, self.max_y,
int(self.max_x * ratio_x),
int(self.max_y * ratio_y)))
self._logger.info('Mouse up: Show ({}, {}) / Capture ({}, {})'.format(self.max_x, self.max_y,
int(self.max_x * ratio_x),
int(self.max_y * ratio_y)))
if self.min_x > self.max_x:
self.min_x, self.max_x = self.max_x, self.min_x
if self.min_y > self.max_y:
self.min_y, self.max_y = self.max_y, self.min_y
self.camera.saveCapture(crop=1,
crop_ax=[
int(self.min_x * ratio_x), int(self.min_y * ratio_x),
int(self.max_x * ratio_x), int(self.max_y * ratio_x)])
t = 0
self.after(250, self.delete('SelectArea'))
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
def setFps(self, fps):
# self.next_frames = int(16 * (60 / int(fps)))
self.next_frames = int(1000 / int(fps))
self._logger.info(f"FPS set to {fps}")
def setShowsize(self, show_height, show_width):
self.show_width = int(show_width)
self.show_height = int(show_height)
self.show_size = (self.show_width, self.show_height)
self.config(width=self.show_width, height=self.show_height)
print("Show size set to {0} x {1}".format(self.show_width, self.show_height))
self._logger.info("Show size set to {0} x {1}".format(self.show_width, self.show_height))
def mouseCtrlLeftPress(self, event):
##_img = cv2.cvtColor(self.camera.image_bgr, cv2.COLOR_BGR2RGB)
image_bgr = self.camera.readFrame()
_img = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
x, y = event.x, event.y
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(x, y, int(x * ratio_x), int(y * ratio_y)))
print(f"Color [R: {_img[int(y * ratio_y), int(x * ratio_x)][0]}, "
f"G: {_img[int(y * ratio_y), int(x * ratio_x)][1]}, "
f"B: {_img[int(y * ratio_y), int(x * ratio_x)][2]}]")
self._logger.info(
'Mouse down: Show ({}, {}) / Capture ({}, {})'.format(x, y, int(x * ratio_x), int(y * ratio_y)))
def mouseCtrlLeftRelease(self, event):
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
def mouseLeftPress(self, event, ser):
if self.master.is_use_right_stick_mouse.get():
self.UnbindRightClick()
self.config(cursor='dot')
self.lx_init, self.ly_init = event.x, event.y
self.lcircle = self.create_oval(self.lx_init - self.radius, self.ly_init - self.radius,
self.lx_init + self.radius, self.ly_init + self.radius,
outline='cyan', tag="lcircle")
self.lcircle2 = self.create_oval(self.lx_init - self.radius // 10, self.ly_init - self.radius // 10,
self.lx_init + self.radius // 10, self.ly_init + self.radius // 10,
fill="cyan", tag="lcircle2")
# self.LStick = StickCommand.StickLeft()
# self.LStick.start(ser)
if isTakeLog:
if self.dq is None:
self.dq = deque()
else:
self.dq.clear()
if self.calc_time is None:
self.calc_time = time.perf_counter()
else:
# LSTICK_logger.debug(f"{0},{0},{time.perf_counter() - self.calc_time}")
self.dq.append([0, 0, time.perf_counter() - self.calc_time])
self._langle = None
self._lmag = None
def mouseLeftPressing(self, event, ser, angle=0):
# _time = self.calc_time
langle = np.rad2deg(np.arctan2(self.ly_init - event.y, event.x - self.lx_init))
mag = np.sqrt((self.ly_init - event.y) ** 2 + (event.x - self.lx_init) ** 2) / self.radius
if mag <= 0:
mag = 0
elif mag >= 1:
mag = 1
if (self._langle and self._lmag) is not None and isTakeLog:
_time = time.perf_counter()
if _time - self.calc_time > 0.05:
# thread_1 = threading.Thread(target=self.LStick.LStick,
# args=(langle,),
# kwargs={'r': mag, 'duration': _time - self.calc_time})
# thread_1.start()
self.ser.writeRow(
f'3 8 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(langle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(langle))))} '
f'80 80',
is_show=False
)
self.dq.append([langle,
mag,
_time - self.calc_time])
self.calc_time = _time
elif not isTakeLog:
self.ser.writeRow(
f'3 8 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(langle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(langle))))}'
f' 80 80',
is_show=False
)
if mag >= 1:
center_x = (self.radius + self.radius // 11) * np.cos(np.deg2rad(langle))
center_y = (self.radius + self.radius // 11) * np.sin(np.deg2rad(langle))
circ_x_1 = self.lx_init + center_x - self.radius // 10
circ_x_2 = self.lx_init + center_x + self.radius // 10
circ_y_1 = self.ly_init - center_y - self.radius // 10
circ_y_2 = self.ly_init - center_y + self.radius // 10
else:
circ_x_1 = event.x - self.radius // 10
circ_x_2 = event.x + self.radius // 10
circ_y_1 = event.y - self.radius // 10
circ_y_2 = event.y + self.radius // 10
self.coords('lcircle2', circ_x_1, circ_y_1, circ_x_2, circ_y_2, )
self._langle = langle
self._lmag = mag
def mouseLeftRelease(self, ser):
self.config(cursor='tcross')
self.ser.writeRow(
f'3 8 80 80',
is_show=False
)
self.delete("lcircle")
self.delete("lcircle2")
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
# self.event_generate('<Motion>', warp=True, x=self.lx_init, y=self.ly_init)
if isTakeLog:
self.dq.append([self._langle,
self._lmag,
time.perf_counter() - self.calc_time])
for _ in self.dq:
self.LSTICK_logger.debug(",".join(list(map(str, _))))
def mouseRightPress(self, event, ser):
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
self.config(cursor='dot')
self.rx_init, self.ry_init = event.x, event.y
self.rcircle = self.create_oval(self.rx_init - self.radius, self.ry_init - self.radius,
self.rx_init + self.radius, self.ry_init + self.radius,
outline='red', tag="rcircle")
self.rcircle2 = self.create_oval(self.rx_init - self.radius // 10, self.ry_init - self.radius // 10,
self.rx_init + self.radius // 10, self.ry_init + self.radius // 10,
fill="red", tag="rcircle2")
# self.RStick = StickCommand.StickRight()
# self.RStick.start(ser)
if isTakeLog:
if self.dq is None:
self.dq = deque()
else:
self.dq.clear()
if self.calc_time is None:
self.calc_time = time.perf_counter()
else:
# LSTICK_logger.debug(f"{0},{0},{time.perf_counter() - self.calc_time}")
self.dq.append([0, 0, time.perf_counter() - self.calc_time])
self._rangle = None
self._rmag = None
def mouseRightPressing(self, event, ser, angle=0):
rangle = np.rad2deg(np.arctan2(self.ry_init - event.y, event.x - self.rx_init))
mag = np.sqrt((self.ry_init - event.y) ** 2 + (event.x - self.rx_init) ** 2) / self.radius
if mag <= 0:
mag = 0
elif mag >= 1:
mag = 1
if (self._langle and self._lmag) is not None and isTakeLog:
_time = time.perf_counter()
if _time - self.calc_time > 0.05:
# thread_1 = threading.Thread(target=self.RStick.RStick,
# args=(rangle,),
# kwargs={'r': mag, 'duration': _time - self.calc_time})
# thread_1.start()
# self.RStick.RStick(rangle, r=mag)
self.ser.writeRow(
f'3 8 80 80 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(rangle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(rangle))))}',
is_show=False
)
self.dq.append([rangle, mag, _time - self.calc_time])
self.calc_time = _time
elif not isTakeLog:
self.ser.writeRow(
f'3 8 80 80 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(rangle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(rangle))))}',
is_show=False
)
if mag >= 1:
center_x = (self.radius + self.radius // 11) * np.cos(np.deg2rad(rangle))
center_y = (self.radius + self.radius // 11) * np.sin(np.deg2rad(rangle))
circ_x_1 = self.rx_init + center_x - self.radius // 10
circ_x_2 = self.rx_init + center_x + self.radius // 10
circ_y_1 = self.ry_init - center_y - self.radius // 10
circ_y_2 = self.ry_init - center_y + self.radius // 10
else:
circ_x_1 = event.x - self.radius // 10
circ_x_2 = event.x + self.radius // 10
circ_y_1 = event.y - self.radius // 10
circ_y_2 = event.y + self.radius // 10
self.coords('rcircle2', circ_x_1, circ_y_1, circ_x_2, circ_y_2, )
self._rangle = rangle
self._rmag = mag
def mouseRightRelease(self, ser):
self.config(cursor='tcross')
self.ser.writeRow(
f'3 8 80 80 80 80',
is_show=False
)
self.delete("rcircle")
self.delete("rcircle2")
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
# self.event_generate('<Motion>', warp=True, x=self.rx_init, y=self.ry_init)
if isTakeLog:
self.dq.append([self._rangle,
self._rmag,
time.perf_counter() - self.calc_time])
for _ in self.dq:
self.RSTICK_logger.debug(",".join(list(map(str, _))))
def startCapture(self):
self.capture()
def capture(self):
if self.is_show_var.get():
image_bgr = self.camera.readFrame()
else:
self.after(self.next_frames, self.capture)
return
if image_bgr is not None:
image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
image_pil = Image.fromarray(image_rgb).resize(self.show_size)
image_tk = ImageTk.PhotoImage(image_pil)
self.im = image_tk
# self.configure( image=image_tk)
self.itemconfig(self.im_, image=image_tk)
else:
self.im = self.disabled_tk
# self.configure(image=self.disabled_tk)
self.itemconfig(self.im_, image=self.disabled_tk)
self.after(self.next_frames, self.capture)
def saveCapture(self):
self.camera.saveCapture()
def ImgRect(self, x1, y1, x2, y2, outline, tag, ms):
ratio_x = float(self.show_size[0] / self.camera.capture_size[0])
ratio_y = float(self.show_size[1] / self.camera.capture_size[1])
self.create_rectangle((x1 - 1.0) * ratio_x, (y1 - 1.0) * ratio_y, (x2 + 1.0) * ratio_x, (y2 + 1.0) * ratio_y,
width=4.5,
outline="white", tag=tag)
self.create_rectangle(x1 * ratio_x, y1 * ratio_y, x2 * ratio_x, y2 * ratio_y, width=2.5,
outline=outline, tag=tag)
self.after(ms, self.deleteImageRect, tag)
def deleteImageRect(self, tag):
self.delete(tag)
def BindLeftClick(self):
self.bind("<ButtonPress-1>", lambda ev: self.mouseLeftPress(ev, self.ser))
self.bind("<Button1-Motion>", lambda ev: self.mouseLeftPressing(ev, self.ser))
self.bind("<ButtonRelease-1>", lambda ev: self.mouseLeftRelease(self.ser))
self._logger.debug("Bind <ButtonPress-1>")
self._logger.debug("Bind <Button1-Motion>")
self._logger.debug("Bind <ButtonRelease-1>")
def BindRightClick(self):
self.bind("<ButtonPress-3>", lambda ev: self.mouseRightPress(ev, self.ser))
self.bind("<Button3-Motion>", lambda ev: self.mouseRightPressing(ev, self.ser))
self.bind("<ButtonRelease-3>", lambda ev: self.mouseRightRelease(self.ser))
self._logger.debug("Bind <ButtonPress-3>")
self._logger.debug("Bind <Button3-Motion>")
self._logger.debug("Bind <ButtonRelease-3>")
def UnbindLeftClick(self):
self.unbind("<ButtonPress-1>")
self.unbind("<Button1-Motion>")
self.unbind("<ButtonRelease-1>")
self._logger.debug("Unbind <ButtonPress-1>")
self._logger.debug("Unbind <Button1-Motion>")
self._logger.debug("Unbind <ButtonRelease-1>")
def UnbindRightClick(self):
self.unbind("<ButtonPress-3>")
self.unbind("<Button3-Motion>")
self.unbind("<ButtonRelease-3>")
self._logger.debug("Unbind <ButtonPress-3>")
self._logger.debug("Unbind <Button3-Motion>")
self._logger.debug("Unbind <ButtonRelease-3>")
# GUI of switch controller simulator
class ControllerGUI:
def __init__(self, root, ser):
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
self.window = tk.Toplevel(root)
self.window.title('Switch Controller Simulator')
self.window.geometry("%dx%d%+d%+d" % (600, 300, 250, 125))
self.window.resizable(0, 0)
joycon_L_color = '#95f1ff'
joycon_R_color = '#ff6b6b'
joycon_L_frame = tk.Frame(self.window, width=300, height=300, relief='flat', bg=joycon_L_color)
joycon_R_frame = tk.Frame(self.window, width=300, height=300, relief='flat', bg=joycon_R_color)
hat_frame = tk.Frame(joycon_L_frame, relief='flat', bg=joycon_L_color)
abxy_frame = tk.Frame(joycon_R_frame, relief='flat', bg=joycon_R_color)
# ABXY
tk.Button(abxy_frame, text='A', command=lambda: UnitCommand.A().start(ser)).grid(row=1, column=2)
tk.Button(abxy_frame, text='B', command=lambda: UnitCommand.B().start(ser)).grid(row=2, column=1)
tk.Button(abxy_frame, text='X', command=lambda: UnitCommand.X().start(ser)).grid(row=0, column=1)
tk.Button(abxy_frame, text='Y', command=lambda: UnitCommand.Y().start(ser)).grid(row=1, column=0)
abxy_frame.place(relx=0.2, rely=0.3)
# HAT
tk.Button(hat_frame, text='UP', command=lambda: UnitCommand.UP().start(ser)).grid(row=0, column=1)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.UP_RIGHT().start(ser)).grid(row=0, column=2)
tk.Button(hat_frame, text='RIGHT', command=lambda: UnitCommand.RIGHT().start(ser)).grid(row=1, column=2)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.DOWN_RIGHT().start(ser)).grid(row=2, column=2)
tk.Button(hat_frame, text='DOWN', command=lambda: UnitCommand.DOWN().start(ser)).grid(row=2, column=1)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.DOWN_LEFT().start(ser)).grid(row=2, column=0)
tk.Button(hat_frame, text='LEFT', command=lambda: UnitCommand.LEFT().start(ser)).grid(row=1, column=0)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.UP_LEFT().start(ser)).grid(row=0, column=0)
hat_frame.place(relx=0.2, rely=0.6)
# L side
tk.Button(joycon_L_frame, text='L', width=20, command=lambda: UnitCommand.L().start(ser)).place(x=30, y=30)
tk.Button(joycon_L_frame, text='ZL', width=20, command=lambda: UnitCommand.ZL().start(ser)).place(x=30, y=0)
tk.Button(joycon_L_frame, text='LCLICK', width=7, command=lambda: UnitCommand.LCLICK().start(ser)).place(x=120,
y=120)
tk.Button(joycon_L_frame, text='MINUS', width=5, command=lambda: UnitCommand.MINUS().start(ser)).place(x=220,
y=70)
tk.Button(joycon_L_frame, text='CAP', width=5, command=lambda: UnitCommand.CAPTURE().start(ser)).place(x=200,
y=270)
# R side
tk.Button(joycon_R_frame, text='R', width=20, command=lambda: UnitCommand.R().start(ser)).place(x=120, y=30)
tk.Button(joycon_R_frame, text='ZR', width=20, command=lambda: UnitCommand.ZR().start(ser)).place(x=120, y=0)
tk.Button(joycon_R_frame, text='RCLICK', width=7, command=lambda: UnitCommand.RCLICK().start(ser)).place(x=120,
y=205)
tk.Button(joycon_R_frame, text='PLUS', width=5, command=lambda: UnitCommand.PLUS().start(ser)).place(x=35, y=70)
tk.Button(joycon_R_frame, text='HOME', width=5, command=lambda: UnitCommand.HOME().start(ser)).place(x=50,
y=270)
joycon_L_frame.grid(row=0, column=0)
joycon_R_frame.grid(row=0, column=1)
# button style settings
for button in abxy_frame.winfo_children():
self.applyButtonSetting(button)
for button in hat_frame.winfo_children():
self.applyButtonSetting(button)
for button in [b for b in joycon_L_frame.winfo_children() if type(b) is tk.Button]:
self.applyButtonColor(button)
for button in [b for b in joycon_R_frame.winfo_children() if type(b) is tk.Button]:
self.applyButtonColor(button)
self._logger.debug("Create GUI controller")
def applyButtonSetting(self, button):
button['width'] = 7
self.applyButtonColor(button)
def applyButtonColor(self, button):
button['bg'] = '#343434'
button['fg'] = '#fff'
def bind(self, event, func):
self.window.bind(event, func)
def protocol(self, event, func):
self.window.protocol(event, func)
def focus_force(self):
self.window.focus_force()
def destroy(self):
self.window.destroy()
self._logger.debug("GUI controller destroyed")
# To avoid the error says 'ScrolledText' object has no attribute 'flush'
class MyScrolledText(ScrolledText):
def flush(self):
pass
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import cv2
import os
import time
import tkinter as tk
from tkinter.scrolledtext import ScrolledText
import numpy as np
import datetime
from collections import deque
from PIL import Image, ImageTk
from Commands import UnitCommand
from Commands import StickCommand
from Commands.Keys import Direction, Stick, Button, Direction, KeyPress
import logging
from logging import INFO, StreamHandler, getLogger, DEBUG, NullHandler
from Commands.PythonCommandBase import PythonCommand, StopThread
try:
os.makedirs('log')
except FileExistsError:
pass
isTakeLog = False
# logger_stick = getLogger(__name__)
nowtime = datetime.datetime.fromtimestamp(time.time()).strftime('%Y%m%d_%H%M%S')
# press button at duration times(s)
class MouseStick(PythonCommand):
NAME = 'MOUSEスティック'
def __init__(self):
super().__init__()
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
def do(self):
pass
def stick(self, buttons, duration=0.1, wait=0.1):
self.keys.input(buttons, ifPrint=False)
self.wait(duration)
self.wait(wait)
# press button at duration times(s)
def stickEnd(self, buttons):
self.keys.inputEnd(buttons)
class CaptureArea(tk.Canvas):
def __init__(self, camera, fps, is_show, ser, master=None, show_width=640, show_height=360):
super().__init__(master, borderwidth=0, cursor='tcross', width=show_width, height=show_height)
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
self.master = master
self.radius = 60 # 描画する円の半径
self.camera = camera
# self.show_size = (640, 360)
self.show_width = int(show_width)
self.show_height = int(show_height)
self.show_size = (self.show_width, self.show_height)
self.is_show_var = is_show
self.lx_init, self.ly_init = 0, 0
self.rx_init, self.ry_init = 0, 0
self.min_x, self.min_y = 0, 0
self.max_x, self.max_y = 0, 0
self.keys = None
self.ser = ser
self.lcircle = None
self.lcircle2 = None
self.rcircle = None
self.rcircle2 = None
self.LStick = None
self.RStick = None
self.calc_time = None
self.ss = None
self.dq = None
self._langle = None
self._lmag = None
self._rangle = None
self._rmag = None
self.stick_handler = StreamHandler()
self.stick_logging_level = DEBUG
self.stick_handler.setLevel(self.stick_logging_level)
# self._logger.setLevel(self.stick_logging_level)
# self._logger.addHandler(self.stick_handler)
# self._logger.propagate = False
if isTakeLog:
filename_base = os.path.join("log", f"{nowtime}")
self.LS = logging.FileHandler(filename=f"{filename_base}_LStick.log", encoding='utf-8')
self.LS.setLevel(logging.DEBUG)
self.LSTICK_logger = logging.getLogger("L_STICK")
self.LSTICK_logger.setLevel(logging.DEBUG)
self.LSTICK_logger.addHandler(self.LS)
self.RS = logging.FileHandler(filename=f"{filename_base}_RStick.log", encoding='utf-8')
self.RS.setLevel(logging.DEBUG)
self.RSTICK_logger = logging.getLogger("R_STICK")
self.RSTICK_logger.setLevel(logging.DEBUG)
self.RSTICK_logger.addHandler(self.RS)
# self.circle =
self.setFps(fps)
self.bind("<Control-ButtonPress-1>", self.mouseCtrlLeftPress)
self.bind("<Control-ButtonRelease-1>", self.mouseCtrlLeftRelease)
self.bind("<Control-Shift-ButtonPress-1>", self.StartRangeSS)
self.bind("<Control-Shift-Button1-Motion>", self.MotionRangeSS)
self.bind("<Control-Shift-ButtonRelease-1>", self.ReleaseRangeSS)
# Set disabled image first
disabled_img = cv2.imread("../Images/disabled.png", cv2.IMREAD_GRAYSCALE)
disabled_pil = Image.fromarray(disabled_img)
self.disabled_tk = ImageTk.PhotoImage(disabled_pil)
self.im = self.disabled_tk
# self.configure(image=self.disabled_tk) # labelからキャンバスに変更したので微修正
self.im_ = self.create_image(0, 0, image=self.disabled_tk, anchor=tk.NW)
def ApplyLStickMouse(self):
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
else:
self.UnbindLeftClick()
def ApplyRStickMouse(self):
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
else:
self.UnbindRightClick()
def StartRangeSS(self, event):
##self.ss = self.camera.image_bgr
image_bgr = self.camera.readFrame()
self.ss = image_bgr
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.UnbindRightClick()
self.min_x, self.min_y = event.x, event.y
self.delete('SelectArea')
self.create_rectangle(self.min_x,
self.min_y,
self.min_x + 1,
self.min_y + 1,
outline='red',
tag='SelectArea')
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(self.min_x, self.min_y,
int(self.min_x * ratio_x),
int(self.min_y * ratio_y)))
self._logger.info('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(self.min_x, self.min_y,
int(self.min_x * ratio_x),
int(self.min_y * ratio_y)))
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
def MotionRangeSS(self, event):
if event.x < 0:
self.max_x = 0
else:
self.max_x = min(self.show_width, event.x)
if event.y < 0:
self.max_y = 0
else:
self.max_y = min(self.show_height, event.y)
self.coords('SelectArea', self.min_x, self.min_y, self.max_x + 1, self.max_y + 1)
self.coords('SelectAreaFilled', self.min_x, self.min_y, self.max_x + 1, self.max_y + 1)
def ReleaseRangeSS(self, event):
# self.max_x, self.max_y = event.x, event.y
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse up: Show ({}, {}) / Capture ({}, {})'.format(self.max_x, self.max_y,
int(self.max_x * ratio_x),
int(self.max_y * ratio_y)))
self._logger.info('Mouse up: Show ({}, {}) / Capture ({}, {})'.format(self.max_x, self.max_y,
int(self.max_x * ratio_x),
int(self.max_y * ratio_y)))
if self.min_x > self.max_x:
self.min_x, self.max_x = self.max_x, self.min_x
if self.min_y > self.max_y:
self.min_y, self.max_y = self.max_y, self.min_y
self.camera.saveCapture(crop=1,
crop_ax=[
int(self.min_x * ratio_x), int(self.min_y * ratio_x),
int(self.max_x * ratio_x), int(self.max_y * ratio_x)])
t = 0
self.after(250, self.delete('SelectArea'))
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
def setFps(self, fps):
# self.next_frames = int(16 * (60 / int(fps)))
self.next_frames = int(1000 / int(fps))
self._logger.info(f"FPS set to {fps}")
def setShowsize(self, show_height, show_width):
self.show_width = int(show_width)
self.show_height = int(show_height)
self.show_size = (self.show_width, self.show_height)
self.config(width=self.show_width, height=self.show_height)
print("Show size set to {0} x {1}".format(self.show_width, self.show_height))
self._logger.info("Show size set to {0} x {1}".format(self.show_width, self.show_height))
def mouseCtrlLeftPress(self, event):
##_img = cv2.cvtColor(self.camera.image_bgr, cv2.COLOR_BGR2RGB)
image_bgr = self.camera.readFrame()
_img = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
x, y = event.x, event.y
ratio_x = float(self.camera.capture_size[0] / self.show_size[0])
ratio_y = float(self.camera.capture_size[1] / self.show_size[1])
print('Mouse down: Show ({}, {}) / Capture ({}, {})'.format(x, y, int(x * ratio_x), int(y * ratio_y)))
print(f"Color [R: {_img[int(y * ratio_y), int(x * ratio_x)][0]}, "
f"G: {_img[int(y * ratio_y), int(x * ratio_x)][1]}, "
f"B: {_img[int(y * ratio_y), int(x * ratio_x)][2]}]")
self._logger.info(
'Mouse down: Show ({}, {}) / Capture ({}, {})'.format(x, y, int(x * ratio_x), int(y * ratio_y)))
def mouseCtrlLeftRelease(self, event):
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
def mouseLeftPress(self, event, ser):
if self.master.is_use_right_stick_mouse.get():
self.UnbindRightClick()
self.config(cursor='dot')
self.lx_init, self.ly_init = event.x, event.y
self.lcircle = self.create_oval(self.lx_init - self.radius, self.ly_init - self.radius,
self.lx_init + self.radius, self.ly_init + self.radius,
outline='cyan', tag="lcircle")
self.lcircle2 = self.create_oval(self.lx_init - self.radius // 10, self.ly_init - self.radius // 10,
self.lx_init + self.radius // 10, self.ly_init + self.radius // 10,
fill="cyan", tag="lcircle2")
# self.LStick = StickCommand.StickLeft()
# self.LStick.start(ser)
if isTakeLog:
if self.dq is None:
self.dq = deque()
else:
self.dq.clear()
if self.calc_time is None:
self.calc_time = time.perf_counter()
else:
# LSTICK_logger.debug(f"{0},{0},{time.perf_counter() - self.calc_time}")
self.dq.append([0, 0, time.perf_counter() - self.calc_time])
self._langle = None
self._lmag = None
def mouseLeftPressing(self, event, ser, angle=0):
# _time = self.calc_time
langle = np.rad2deg(np.arctan2(self.ly_init - event.y, event.x - self.lx_init))
mag = np.sqrt((self.ly_init - event.y) ** 2 + (event.x - self.lx_init) ** 2) / self.radius
if mag <= 0:
mag = 0
elif mag >= 1:
mag = 1
if (self._langle and self._lmag) is not None and isTakeLog:
_time = time.perf_counter()
if _time - self.calc_time > 0.05:
# thread_1 = threading.Thread(target=self.LStick.LStick,
# args=(langle,),
# kwargs={'r': mag, 'duration': _time - self.calc_time})
# thread_1.start()
self.ser.writeRow(
f'3 8 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(langle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(langle))))} '
f'80 80',
is_show=False
)
self.dq.append([langle,
mag,
_time - self.calc_time])
self.calc_time = _time
elif not isTakeLog:
self.ser.writeRow(
f'3 8 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(langle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(langle))))}'
f' 80 80',
is_show=False
)
if mag >= 1:
center_x = (self.radius + self.radius // 11) * np.cos(np.deg2rad(langle))
center_y = (self.radius + self.radius // 11) * np.sin(np.deg2rad(langle))
circ_x_1 = self.lx_init + center_x - self.radius // 10
circ_x_2 = self.lx_init + center_x + self.radius // 10
circ_y_1 = self.ly_init - center_y - self.radius // 10
circ_y_2 = self.ly_init - center_y + self.radius // 10
else:
circ_x_1 = event.x - self.radius // 10
circ_x_2 = event.x + self.radius // 10
circ_y_1 = event.y - self.radius // 10
circ_y_2 = event.y + self.radius // 10
self.coords('lcircle2', circ_x_1, circ_y_1, circ_x_2, circ_y_2, )
self._langle = langle
self._lmag = mag
def mouseLeftRelease(self, ser):
self.config(cursor='tcross')
self.ser.writeRow(
f'3 8 80 80',
is_show=False
)
self.delete("lcircle")
self.delete("lcircle2")
if self.master.is_use_right_stick_mouse.get():
self.BindRightClick()
# self.event_generate('<Motion>', warp=True, x=self.lx_init, y=self.ly_init)
if isTakeLog:
self.dq.append([self._langle,
self._lmag,
time.perf_counter() - self.calc_time])
for _ in self.dq:
self.LSTICK_logger.debug(",".join(list(map(str, _))))
def mouseRightPress(self, event, ser):
if self.master.is_use_left_stick_mouse.get():
self.UnbindLeftClick()
self.config(cursor='dot')
self.rx_init, self.ry_init = event.x, event.y
self.rcircle = self.create_oval(self.rx_init - self.radius, self.ry_init - self.radius,
self.rx_init + self.radius, self.ry_init + self.radius,
outline='red', tag="rcircle")
self.rcircle2 = self.create_oval(self.rx_init - self.radius // 10, self.ry_init - self.radius // 10,
self.rx_init + self.radius // 10, self.ry_init + self.radius // 10,
fill="red", tag="rcircle2")
# self.RStick = StickCommand.StickRight()
# self.RStick.start(ser)
if isTakeLog:
if self.dq is None:
self.dq = deque()
else:
self.dq.clear()
if self.calc_time is None:
self.calc_time = time.perf_counter()
else:
# LSTICK_logger.debug(f"{0},{0},{time.perf_counter() - self.calc_time}")
self.dq.append([0, 0, time.perf_counter() - self.calc_time])
self._rangle = None
self._rmag = None
def mouseRightPressing(self, event, ser, angle=0):
rangle = np.rad2deg(np.arctan2(self.ry_init - event.y, event.x - self.rx_init))
mag = np.sqrt((self.ry_init - event.y) ** 2 + (event.x - self.rx_init) ** 2) / self.radius
if mag <= 0:
mag = 0
elif mag >= 1:
mag = 1
if (self._langle and self._lmag) is not None and isTakeLog:
_time = time.perf_counter()
if _time - self.calc_time > 0.05:
# thread_1 = threading.Thread(target=self.RStick.RStick,
# args=(rangle,),
# kwargs={'r': mag, 'duration': _time - self.calc_time})
# thread_1.start()
# self.RStick.RStick(rangle, r=mag)
self.ser.writeRow(
f'3 8 80 80 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(rangle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(rangle))))}',
is_show=False
)
self.dq.append([rangle, mag, _time - self.calc_time])
self.calc_time = _time
elif not isTakeLog:
self.ser.writeRow(
f'3 8 80 80 '
f'{hex(int(128 + mag * 127.5 * np.cos(np.deg2rad(rangle))))} '
f'{hex(int(128 - mag * 127.5 * np.sin(np.deg2rad(rangle))))}',
is_show=False
)
if mag >= 1:
center_x = (self.radius + self.radius // 11) * np.cos(np.deg2rad(rangle))
center_y = (self.radius + self.radius // 11) * np.sin(np.deg2rad(rangle))
circ_x_1 = self.rx_init + center_x - self.radius // 10
circ_x_2 = self.rx_init + center_x + self.radius // 10
circ_y_1 = self.ry_init - center_y - self.radius // 10
circ_y_2 = self.ry_init - center_y + self.radius // 10
else:
circ_x_1 = event.x - self.radius // 10
circ_x_2 = event.x + self.radius // 10
circ_y_1 = event.y - self.radius // 10
circ_y_2 = event.y + self.radius // 10
self.coords('rcircle2', circ_x_1, circ_y_1, circ_x_2, circ_y_2, )
self._rangle = rangle
self._rmag = mag
def mouseRightRelease(self, ser):
self.config(cursor='tcross')
self.ser.writeRow(
f'3 8 80 80 80 80',
is_show=False
)
self.delete("rcircle")
self.delete("rcircle2")
if self.master.is_use_left_stick_mouse.get():
self.BindLeftClick()
# self.event_generate('<Motion>', warp=True, x=self.rx_init, y=self.ry_init)
if isTakeLog:
self.dq.append([self._rangle,
self._rmag,
time.perf_counter() - self.calc_time])
for _ in self.dq:
self.RSTICK_logger.debug(",".join(list(map(str, _))))
def startCapture(self):
self.capture()
def capture(self):
if self.is_show_var.get():
image_bgr = self.camera.readFrame()
else:
self.after(self.next_frames, self.capture)
return
if image_bgr is not None:
image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
image_pil = Image.fromarray(image_rgb).resize(self.show_size)
image_tk = ImageTk.PhotoImage(image_pil)
self.im = image_tk
# self.configure( image=image_tk)
self.itemconfig(self.im_, image=image_tk)
else:
self.im = self.disabled_tk
# self.configure(image=self.disabled_tk)
self.itemconfig(self.im_, image=self.disabled_tk)
self.after(self.next_frames, self.capture)
def saveCapture(self):
self.camera.saveCapture()
def ImgRect(self, x1, y1, x2, y2, outline, tag, ms):
ratio_x = float(self.show_size[0] / self.camera.capture_size[0])
ratio_y = float(self.show_size[1] / self.camera.capture_size[1])
self.create_rectangle((x1 - 1.0) * ratio_x, (y1 - 1.0) * ratio_y, (x2 + 1.0) * ratio_x, (y2 + 1.0) * ratio_y,
width=4.5,
outline="white", tag=tag)
self.create_rectangle(x1 * ratio_x, y1 * ratio_y, x2 * ratio_x, y2 * ratio_y, width=2.5,
outline=outline, tag=tag)
self.after(ms, self.deleteImageRect, tag)
def deleteImageRect(self, tag):
self.delete(tag)
def BindLeftClick(self):
self.bind("<ButtonPress-1>", lambda ev: self.mouseLeftPress(ev, self.ser))
self.bind("<Button1-Motion>", lambda ev: self.mouseLeftPressing(ev, self.ser))
self.bind("<ButtonRelease-1>", lambda ev: self.mouseLeftRelease(self.ser))
self._logger.debug("Bind <ButtonPress-1>")
self._logger.debug("Bind <Button1-Motion>")
self._logger.debug("Bind <ButtonRelease-1>")
def BindRightClick(self):
self.bind("<ButtonPress-3>", lambda ev: self.mouseRightPress(ev, self.ser))
self.bind("<Button3-Motion>", lambda ev: self.mouseRightPressing(ev, self.ser))
self.bind("<ButtonRelease-3>", lambda ev: self.mouseRightRelease(self.ser))
self._logger.debug("Bind <ButtonPress-3>")
self._logger.debug("Bind <Button3-Motion>")
self._logger.debug("Bind <ButtonRelease-3>")
def UnbindLeftClick(self):
self.unbind("<ButtonPress-1>")
self.unbind("<Button1-Motion>")
self.unbind("<ButtonRelease-1>")
self._logger.debug("Unbind <ButtonPress-1>")
self._logger.debug("Unbind <Button1-Motion>")
self._logger.debug("Unbind <ButtonRelease-1>")
def UnbindRightClick(self):
self.unbind("<ButtonPress-3>")
self.unbind("<Button3-Motion>")
self.unbind("<ButtonRelease-3>")
self._logger.debug("Unbind <ButtonPress-3>")
self._logger.debug("Unbind <Button3-Motion>")
self._logger.debug("Unbind <ButtonRelease-3>")
# GUI of switch controller simulator
class ControllerGUI:
def __init__(self, root, ser):
self._logger = getLogger(__name__)
self._logger.addHandler(NullHandler())
self._logger.setLevel(DEBUG)
self._logger.propagate = True
self.window = tk.Toplevel(root)
self.window.title('Switch Controller Simulator')
self.window.geometry("%dx%d%+d%+d" % (600, 300, 250, 125))
self.window.resizable(0, 0)
joycon_L_color = '#95f1ff'
joycon_R_color = '#ff6b6b'
joycon_L_frame = tk.Frame(self.window, width=300, height=300, relief='flat', bg=joycon_L_color)
joycon_R_frame = tk.Frame(self.window, width=300, height=300, relief='flat', bg=joycon_R_color)
hat_frame = tk.Frame(joycon_L_frame, relief='flat', bg=joycon_L_color)
abxy_frame = tk.Frame(joycon_R_frame, relief='flat', bg=joycon_R_color)
# ABXY
tk.Button(abxy_frame, text='A', command=lambda: UnitCommand.A().start(ser)).grid(row=1, column=2)
tk.Button(abxy_frame, text='B', command=lambda: UnitCommand.B().start(ser)).grid(row=2, column=1)
tk.Button(abxy_frame, text='X', command=lambda: UnitCommand.X().start(ser)).grid(row=0, column=1)
tk.Button(abxy_frame, text='Y', command=lambda: UnitCommand.Y().start(ser)).grid(row=1, column=0)
abxy_frame.place(relx=0.2, rely=0.3)
# HAT
tk.Button(hat_frame, text='UP', command=lambda: UnitCommand.UP().start(ser)).grid(row=0, column=1)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.UP_RIGHT().start(ser)).grid(row=0, column=2)
tk.Button(hat_frame, text='RIGHT', command=lambda: UnitCommand.RIGHT().start(ser)).grid(row=1, column=2)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.DOWN_RIGHT().start(ser)).grid(row=2, column=2)
tk.Button(hat_frame, text='DOWN', command=lambda: UnitCommand.DOWN().start(ser)).grid(row=2, column=1)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.DOWN_LEFT().start(ser)).grid(row=2, column=0)
tk.Button(hat_frame, text='LEFT', command=lambda: UnitCommand.LEFT().start(ser)).grid(row=1, column=0)
tk.Button(hat_frame, text='', command=lambda: UnitCommand.UP_LEFT().start(ser)).grid(row=0, column=0)
hat_frame.place(relx=0.2, rely=0.6)
# L side
tk.Button(joycon_L_frame, text='L', width=20, command=lambda: UnitCommand.L().start(ser)).place(x=30, y=30)
tk.Button(joycon_L_frame, text='ZL', width=20, command=lambda: UnitCommand.ZL().start(ser)).place(x=30, y=0)
tk.Button(joycon_L_frame, text='LCLICK', width=7, command=lambda: UnitCommand.LCLICK().start(ser)).place(x=120,
y=120)
tk.Button(joycon_L_frame, text='MINUS', width=5, command=lambda: UnitCommand.MINUS().start(ser)).place(x=220,
y=70)
tk.Button(joycon_L_frame, text='CAP', width=5, command=lambda: UnitCommand.CAPTURE().start(ser)).place(x=200,
y=270)
# R side
tk.Button(joycon_R_frame, text='R', width=20, command=lambda: UnitCommand.R().start(ser)).place(x=120, y=30)
tk.Button(joycon_R_frame, text='ZR', width=20, command=lambda: UnitCommand.ZR().start(ser)).place(x=120, y=0)
tk.Button(joycon_R_frame, text='RCLICK', width=7, command=lambda: UnitCommand.RCLICK().start(ser)).place(x=120,
y=205)
tk.Button(joycon_R_frame, text='PLUS', width=5, command=lambda: UnitCommand.PLUS().start(ser)).place(x=35, y=70)
tk.Button(joycon_R_frame, text='HOME', width=5, command=lambda: UnitCommand.HOME().start(ser)).place(x=50,
y=270)
joycon_L_frame.grid(row=0, column=0)
joycon_R_frame.grid(row=0, column=1)
# button style settings
for button in abxy_frame.winfo_children():
self.applyButtonSetting(button)
for button in hat_frame.winfo_children():
self.applyButtonSetting(button)
for button in [b for b in joycon_L_frame.winfo_children() if type(b) is tk.Button]:
self.applyButtonColor(button)
for button in [b for b in joycon_R_frame.winfo_children() if type(b) is tk.Button]:
self.applyButtonColor(button)
self._logger.debug("Create GUI controller")
def applyButtonSetting(self, button):
button['width'] = 7
self.applyButtonColor(button)
def applyButtonColor(self, button):
button['bg'] = '#343434'
button['fg'] = '#fff'
def bind(self, event, func):
self.window.bind(event, func)
def protocol(self, event, func):
self.window.protocol(event, func)
def focus_force(self):
self.window.focus_force()
def destroy(self):
self.window.destroy()
self._logger.debug("GUI controller destroyed")
# To avoid the error says 'ScrolledText' object has no attribute 'flush'
class MyScrolledText(ScrolledText):
def flush(self):
pass
変更後、Poke-Controllerの再起動を実行してください。
以上でラズベリーパイにPoke-Controllerを導入する手順は終了です。
Poke-Controller上で動作させるpythonプログラムについては各自修正してください。
必要に応じてラズベリーパイ版Visual Studio Codeと画像編集のためにGIMPをインストールしておくと便利です。
sudo apt install code
sudo apt install gimp
pokecon poke-controller ぽけこん ポケコン ポケコントローラー NX NX2 NX Macro Controller 2 ラズパイ ラズベリーパイ raspberry pi
いいなと思ったら応援しよう！
