For my Uni thesis project, I need firstly to setup automatically multiple github repositories within python script. Problems I am facing is that:
- each repository has its own requirements and some of them conflict with other repository's requirements. The most prominent is 'diffusers', for one method I am using huggingface's diffusers for other method I am using ShivamShrirao's one. but when importing, it it is only import diffusers, so I think when I 'pip install' one of them it is overwritting the other one.
- I need to call functions from these repositories inside the code, so I think I need to add the cloned repositories to PATH.
so for that I created a folder structure as follows:
main
├── method_1_folder
│ ├── method_1_github_repository_cloned_folder
│ └── venv
├── method_2_folder
│ ├── method_2_github_repository_cloned_folder
│ └── venv
└── running_python_file.py
and tried to create a separate environment for each method so I would call a prep_work function that would clone the github, create an environment, activate it, then add its path to python path then install all requirements. makes sense right? but when I print the path of the active environment after all the actions above, I find out that I am still working on the running_python_file.py environment. would you please help to correct whatever I am doing wrong?
def prep_work(filename, repo_file_name, repo_url, characters, trainingstep=None):
environment_not_created = True
# cloning respository
path = os.path.join(os.getcwd(), filename, repo_file_name)
if not os.path.exists(path):
Repo.clone_from(repo_url, path)
subprocess.run(["python", "-m", "venv", f"{os.path.join(os.getcwd(), filename, 'venv')}"], check=True)
environment_not_created = False
os.chdir(os.path.join(os.getcwd(), filename))
os.system(f'source {os.path.join(os.getcwd(), "venv", "bin", "activate")}')
sys.path.append(path)
print("Active environment:", sys.prefix)
running_python_filebut you if you could switch the logic in it to a bash script, then your issue is fixed (in theory)bin/pythonexecutable. Use that executable in the sub-processes.subprocess.check_call(['path/to/venv/bin/python', 'some_script.py']).