Error when running subprocess.call scrapy

[Huy_D_Quang (Huy Đỗ Quang)]

I’m building a flatpak builded project. I have a button, when it is clicked I want it to run a scrapy script to crawl data.
window.ui

<object class="GtkButton" id="reload_btn">
                            <property name="label" translatable="yes">Reload data</property>
                            <signal name="clicked" handler="on_reload_btn_clicked" swapped="no"/>
</object>

window.py

import subprocess
@Gtk.Template.Callback()
    def on_reload_btn_clicked(self,button):
         name = "province"
         subprocess.call(["scrapy",'crawl','province','-O /home/huydq/Projects/CovidTracking/provinces/province.json'],cwd='/home/huydq/Projects/CovidTracking/provinces')

The structure of project is like:

/home/huydq/Projects/CovidTracking
└───provinces
│   └───spiders
│       │   province.py
│   └───province.json
└───src
    │   window.py
    │   window.ui

I tried to make a test.py file to run subprocess.call with the above command, it works fine.
But when I run project it gets following error:

Traceback (most recent call last):
  File "/app/share/covidtracking/covidtracking/window.py", line 88, in on_reload_btn_clicked
    subprocess.call(["scrapy",'crawl','province','-O /home/huydq/Projects/CovidTracking/src/provinces/province.json'],cwd='/home/huydq/Projects/CovidTracking/src/provinces')
  File "/usr/lib/python3.9/subprocess.py", line 349, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'scrapy'
Application exited

[ariktaurendil (Andrés Fernández)]

The last line of the log it saying what’s wrong. It says that it couldn’t find a file called scrapy. Try to give the function the absolute path to the script.

[Huy_D_Quang (Huy Đỗ Quang)]

I add shell=True and in runtime I get error: “crawl: line 1: scrapy: command not found”
It appears I don’t import scrapy yet so I add the following in my requirements.txt:
cryptography
pyOpenSSL==21.0.0
Scrapy==2.5.1
scrapy-splash==0.8.0
setuptools-scm==6.3.2
And when I run it get following errors:
Failed to build cryptography
ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
Error: module python3-cryptography: Child process exited with code 1