0

Here is my ipython_config.py

$ cat ~/.ipython/profile_default/ipython_config.py
c = get_config()
c.InteractiveShellApp.exec_lines = [
    'import sys',
    'sys.path.append("/code")',
    'import pandas as pd',
]

Here is my file structure

# root @ 49a10d67a5d5 in /code on git:main x [15:12:53] 
$ tree
.
|-- errata
|   |-- test.ipynb
|   `-- top_funds.py
|-- skew_all.csv
`-- test.ipynb

1 directory, 4 files

Here is my ~/.profile grep

# root @ 49a10d67a5d5 in /code on git:main x [16:00:59] 
$ cat ~/.profile| grep PY
export PYTHONPATH=$PYTHONPATH:/code
export JUPYTER_PATH=$JUPYTER_PATH:/code

I want to read file /code/skew_all.csv and import modules in /code. Things are strange.

  1. If I use ipython in the terminal with working directory /code, I can easily read it and import modules in /code.
# root @ 49a10d67a5d5 in /code on git:main x [14:42:01] 
$ ipython
Python 3.8.10 (default, Mar 13 2023, 10:26:41) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.12.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sys

In [2]: sys.path
Out[2]: 
['/usr/local/bin',
 '/code',
 '/usr/lib/python38.zip',
 '/usr/lib/python3.8',
 '/usr/lib/python3.8/lib-dynload',
 '',
 '/usr/local/lib/python3.8/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/code']

In [3]: pd.read_csv('skew_all.csv')
Out[3]: 
           date  skew_510300
0    2019-12-23    -0.097785
1    2019-12-24    -0.091429
2    2019-12-25    -0.111135
3    2019-12-26    -0.104044
4    2019-12-27    -0.151558
..          ...          ...
808  2023-04-26     0.035485
809  2023-04-27     0.008492
810  2023-04-28     0.024342
811  2023-05-04     0.049912
812  2023-05-05     0.030516

[813 rows x 2 columns]

In [4]: from errata.top_funds import *

In [5]:
  1. If I use ipython in the terminal with working directory /code/errata, I can NOT read it BUT can import modules in /code.
# root @ 49a10d67a5d5 in /code/errata on git:main x [14:46:20] 
$ ipython
Python 3.8.10 (default, Mar 13 2023, 10:26:41) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.12.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sys

In [2]: sys.path
Out[2]: 
['/usr/local/bin',
 '/code/errata',
 '/code',
 '/usr/lib/python38.zip',
 '/usr/lib/python3.8',
 '/usr/lib/python3.8/lib-dynload',
 '',
 '/usr/local/lib/python3.8/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/code']

In [3]: pd.read_csv('skew_all.csv')
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[3], line 1
----> 1 pd.read_csv('skew_all.csv')
many tracebacks
FileNotFoundError: [Errno 2] No such file or directory: 'skew_all.csv'

In [4]: from errata.top_funds import *

In [5]:
  1. If I use Jupyter notebook in VS Code with working directory /code, I can NOT read it too, BUT can import modules in /code. This is strange and unusual. But can be solved by setting Notebook File Root to ${workspaceFolder} thanks to @MingJie-MSFT

This file is /code/test.ipynb and screenshot was took with working directory of /code.

enter image description here

  1. If I use Jupyter notebook in VS Code with working directory /code/errata, I can NOT read it too, BUT can import modules in /code.

And this can NOT be solved by setting Notebook File Root to ${workspaceFolder}, with working dir of /code/errata, but can be solved with working dir of /code.

This file is /code/errata/test.ipynb and screenshot was took with working directory of /code.

enter image description here

So the basic conclusion is path is working because of the successful module import, but read_csv not.

I guess there is something unclear with pandas.read_csv path handling. And probably some unknown difference between Jupyter notebook in VS Code and terminal ipython

My goal is to run pd.read_csv('skew_all.csv') and from errata.top_funds import * smoothly without writing any addition codes. The solution I've taken is the ipython_config.py but it has the above strange things.

Improve this question
5
  • An additional constrain, I don't want to write code with full path like read_csv('/code/skew_all.csv') Commented May 17, 2023 at 7:00
  • An additional information, if I remove the ~/.ipython/profile_default/ipython_config.py, Jupyter notebook in VS Code with working dir of /code can read the .csv as normal. Commented May 17, 2023 at 7:04
  • Can you provide a simple directory structure? Would you mind using the .env file and using absolute paths in the env file at the same time. Commented May 17, 2023 at 7:07
  • sure, I've added the simplified file structure in 5. Commented May 17, 2023 at 7:13
  • # root @ 49a10d67a5d5 in /code on git:main x [15:16:16] C:1 $ env | grep PY PYTHONPATH=:/code JUPYTER_PATH=:/code Commented May 17, 2023 at 7:16

1 Answer 1

Reset to default
1

You can refer to this answer.

Open your settings and search for Notebook File Root

Then change it to ${workspaceFolder} and reload VSCode.

enter image description here

It seems that after updating, it runs based on the directory where the file is located as the root directory.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

well, I set it to /code or ${workspaceFolder} both solved the condition 3&4. But disadvantages are 1. have to specify this settings in this workspace, this is an extra procedure 2. run ipython in /code/errata won't read the `.csv
I believe it is correct that ipython cannot read because they are not in the same directory unless you import an absolute path in the file or add the file path to the environment variable by sys.path.append("..").
For your first point of view, I believe it is not a problem. If you do not modify certain settings, it will indeed not meet your needs. Moreover, changing it to ${workspaceFolder} is consistent with the settings of the .py file. Python scripts in VSCode also use the workspace as the root directory.
The ~/.ipython/profile_default/ipython_config.py have solved the path adding problem. Every time a new ipython process started, it executes 'sys.path.append("/code")'. The import sys, sys.path has shown the correct path added. But it just doesn't work, I'm so confused.
Emmm, in fact this set of code is used by a team and everyone set it again, then explain this to them is hard. The docker method was token to make this set of code easy-to-use for everyone, and the ~/.ipython/profile_default/ipython_config.py was set when constructing container.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.