Jedi autocomplete stuck in loading state (Need help replicating) #5514

Sharpz7 · 2022-08-28T22:39:40Z

Sharpz7
Aug 28, 2022

My Jedi autocomplete/hints doesn't seem to want to work with the fastai library, but I imagine its more to do with how huge it is / the use of wildcard imports.

I am trying to figure out if:

This is a problem of my machine now being fast enough
This is an issue with my install
This is an issue with my extentions.
Or if this is to do with code-server

Thanks in advance! If someone else could see if they can replicate this issue that would be great. See the code below:

from time import sleep

from duckduckgo_search import ddg_images
from fastai.vision.all import *
from fastcore.all import *
from fastdownload import download_url

PATH = Path("./files/classifier/")
TEST_PATH = "./files/classifier/test/"
LEARNER = "bird_classifier.pkl"


def search_images(term, max_images=30):
    print(f"Searching for '{term}'")
    return L(ddg_images(term, max_results=max_images)).itemgot("image")


def build():
    searches = "forest", "bird", "plane"

    for o in searches:
        dest = PATH / o
        dest.mkdir(exist_ok=True, parents=True)
        download_images(dest, urls=search_images(f"{o} photo"))
        download_images(dest, urls=search_images(f"{o} sun photo"))
        download_images(dest, urls=search_images(f"{o} shade photo"))
        resize_images(PATH / o, max_size=400, dest=PATH / o)

    failed = verify_images(get_image_files(PATH))
    failed.map(Path.unlink)
    len(failed)

    dls = DataBlock(
        blocks=(ImageBlock, CategoryBlock),
        get_items=get_image_files,
        splitter=RandomSplitter(valid_pct=0.2, seed=42),
        get_y=parent_label,
        item_tfms=[Resize(192, method="squish")],
    ).dataloaders(PATH, bs=32)

    learn = vision_learner(dls, resnet18, metrics=error_rate)
    learn.fine_tune(3)

    return learn


def classifier_test(learn):
    urls = search_images("bird photos", max_images=1)
    urls[0]

    dest = TEST_PATH + "/bird.jpg"
    download_url(urls[0], dest, show_progress=False)

    is_bird, _, probs = learn.predict(PILImage.create(dest))
    print(f"This is a: {is_bird}.")
    print(f"Probability it's a bird: {probs[2]:.4f}")


if __name__ == "__main__":
    # if LEARNER exists, skip building
    if not Path(LEARNER).exists():
        learn = build()
        learn.save(LEARNER, with_opt=True)

    learn = load_learner(LEARNER)
    classifier_test(learn)

And the script to run it

pip install fastai duckduckgo-search
python3 classifier.py

Answered by Sharpz7

Aug 30, 2022

Okay so a final topic on this. Jedi is extremely slow, and doesn't manage to handle wildcards in this case. Pylance can, but by default PyRight (The open source base of Pylance), it doesn't work.

The difference is a single setting that Pylance has true by default, but Pyright does not. I think purely due to the fact that the Pyright devs have a grub against this feature ;/

    "python.analysis.useLibraryCodeForTypes": true,

After enabling this, pyright works just like pylance, and you get full highlighting and autocomplete support for wildcard imports. Data Analysts rejoice :))

View full answer

jsjoeio · 2022-08-29T16:58:55Z

jsjoeio
Aug 29, 2022

Apologies, I do not use Python regularly so might be doing something wrong. I'm testing this in my Terminal on macOS. I ran:

pip3 install fastai duckduckgo-search
python3 classifier.py

and now I'm getting:

Traceback (most recent call last):
  File "classifier.py", line 3, in <module>
    from duckduckgo_search import ddg_images
ModuleNotFoundError: No module named 'duckduckgo_search'

When I tried running pip install fastai duckduckgo-search, I got:

Traceback (most recent call last):
  File "/usr/local/bin/pip", line 11, in <module>
    load_entry_point('pip==21.0.1', 'console_scripts', 'pip')()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2843, in load_entry_point
    return ep.load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2434, in load
    return self.resolve()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2440, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Library/Python/2.7/site-packages/pip-21.0.1-py2.7.egg/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

I followed this but still no luck. Happy to keep trying if you have other tips.

0 replies

Sharpz7 · 2022-08-29T17:36:44Z

Sharpz7
Aug 29, 2022
Author

Huh that is interesting. could you try python3 -m pip install fastai and then python3 -m pip install duckduckgo-search?

Also after further debugging, the Jedi typehint info works flawlessly before vscode fully loads, then starts to lag. This makes me think its a code-server problem. I am on 4.5.1 so I am going to try updating and see if that helps.

3 replies

jsjoeio Aug 29, 2022

That got me past the first error! The script started running then stopped:

❯ python3 classifier.py 
Searching for 'forest photo'
Searching for 'forest sun photo'
Searching for 'forest shade photo'
Searching for 'bird photo'
Searching for 'bird sun photo'
Searching for 'bird shade photo'
Searching for 'plane photo'
Searching for 'plane sun photo'
Searching for 'plane shade photo'
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/_utils.py:209: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
  f"The parameter '{pretrained_param}' is deprecated since 0.13 and will be removed in 0.15, "
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /Users/jp/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1350, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1277, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1323, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1272, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1032, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 972, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1447, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "classifier.py", line 62, in <module>
    learn = build()
  File "classifier.py", line 41, in build
    learn = vision_learner(dls, resnet18, metrics=error_rate)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/fastai/vision/learner.py", line 225, in vision_learner
    model = create_vision_model(arch, n_out, pretrained=pretrained, **model_args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/fastai/vision/learner.py", line 164, in create_vision_model
    model = arch(pretrained=pretrained)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/_utils.py", line 142, in wrapper
    return fn(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/_utils.py", line 228, in inner_wrapper
    return builder(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/resnet.py", line 670, in resnet18
    return _resnet(BasicBlock, [2, 2, 2, 2], weights, progress, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/resnet.py", line 301, in _resnet
    model.load_state_dict(weights.get_state_dict(progress=progress))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torchvision/models/_api.py", line 63, in get_state_dict
    return load_state_dict_from_url(self.url, progress=progress)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/hub.py", line 727, in load_state_dict_from_url
    download_url_to_file(url, cached_file, hash_prefix, progress=progress)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/hub.py", line 593, in download_url_to_file
    u = urlopen(req)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1393, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1352, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)>

Sharpz7 Aug 29, 2022
Author

Hmm that totally stumps me. Maybe because its python3.7? I am testing on Python3.9

jsjoeio Aug 29, 2022

Oh maybe! Yeah, I'm on Python 3.7.9

Sharpz7 · 2022-08-29T20:13:50Z

Sharpz7
Aug 29, 2022
Author

1 reply

jsjoeio Aug 29, 2022

Hooray!

Sharpz7 · 2022-08-30T00:07:00Z

Sharpz7
Aug 30, 2022
Author

So I thought the problem was fixed, but it seems it was only half fixed, i.e the wildcard imports don't work. I am now wondering if this is just a limitaition of Jedi vs pylance

2 replies

Sharpz7 Aug 30, 2022
Author

Using ssh to a remote EC2 instance seems to back this up. When I switch to Jedi the same thing happens https://files.mcaq.me/8kbar.jpg

I never knew pyright existed, so I guess I will switch to that :)))

jsjoeio Aug 30, 2022

interesting! so is that the solution? switching to pyright?

Sharpz7 · 2022-08-30T17:19:27Z

Sharpz7
Aug 30, 2022
Author

Okay so a final topic on this. Jedi is extremely slow, and doesn't manage to handle wildcards in this case. Pylance can, but by default PyRight (The open source base of Pylance), it doesn't work.

The difference is a single setting that Pylance has true by default, but Pyright does not. I think purely due to the fact that the Pyright devs have a grub against this feature ;/

    "python.analysis.useLibraryCodeForTypes": true,

After enabling this, pyright works just like pylance, and you get full highlighting and autocomplete support for wildcard imports. Data Analysts rejoice :))

0 replies

Sharpz7 · 2022-10-11T07:26:59Z

Sharpz7
Oct 11, 2022
Author

It seems to half be the issue. Pyright still doesn't correctly handle the wildcard stuff but the pyright team think that might be a bug / because pylance has better compiled stubs. Obviously testing this is difficult as I cannot test pylance in code-server but I shall report back my findings.

…

On Tue, 30 Aug 2022, 15:53 Joe Previte, ***@***.***> wrote: interesting! so is that the solution? switching to pyright? — Reply to this email directly, view it on GitHub <#5514 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALCTWHTYFO3Z57LJGZIVZN3V3YN7FANCNFSM573Z55TQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jedi autocomplete stuck in loading state (Need help replicating) #5514

{{title}}

Replies: 6 comments 6 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Jedi autocomplete stuck in loading state (Need help replicating) #5514

Sharpz7 Aug 28, 2022

Replies: 6 comments · 6 replies

jsjoeio Aug 29, 2022

Sharpz7 Aug 29, 2022 Author

jsjoeio Aug 29, 2022

Sharpz7 Aug 29, 2022 Author

jsjoeio Aug 29, 2022

Sharpz7 Aug 29, 2022 Author

jsjoeio Aug 29, 2022

Sharpz7 Aug 30, 2022 Author

Sharpz7 Aug 30, 2022 Author

jsjoeio Aug 30, 2022

Sharpz7 Aug 30, 2022 Author

Sharpz7 Oct 11, 2022 Author

Sharpz7
Aug 28, 2022

Replies: 6 comments 6 replies

jsjoeio
Aug 29, 2022

Sharpz7
Aug 29, 2022
Author

Sharpz7 Aug 29, 2022
Author

Sharpz7
Aug 29, 2022
Author

Sharpz7
Aug 30, 2022
Author

Sharpz7 Aug 30, 2022
Author

Sharpz7
Aug 30, 2022
Author

Sharpz7
Oct 11, 2022
Author