python - Meta llama 3.2 3b model local download - Stack Overflow

admin管理员组

文章数量:1389875

I am installed the llamma3.2 model from meta directly and got it in this format

-a----         3/10/2025   3:22 PM            209 checklist.chk
-a----         3/10/2025   9:47 AM     6425585114 consolidated.00.pth
-a----         3/10/2025   9:27 AM            220 params.json
-a----         3/10/2025   9:27 AM        2183982 tokenizer.model

And I am trying to run it, but it is giving me this error

Traceback (most recent call last):
  File "C:\Users\user\.llama\checkpoints\Llama3.2-3B-Instruct\LLAMA.PY", line 14, in <module>
    model = LlamaForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 262, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 3808, in from_pretrained       
    raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in director

running this code

# Install required packages if you haven't already
# pip install torch transformers

import torch
from transformers import LlamaForCausalLM, LlamaTokenizer

# Path to your model files
model_path = "C:/Users/user/.llama/checkpoints/Llama3.2-3B-Instruct"  # The folder containing the files you listed

# Load the tokenizer
tokenizer = LlamaTokenizer.from_pretrained(model_path)

# Load the model
model = LlamaForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.float16,  # Use float16 for efficiency
    low_cpu_mem_usage=True,     # Helps with memory usage
    device_map="auto"           # Automatically use GPU if available
)

# Generate text
def generate_text(prompt, max_length=100):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    # Generate
    with torch.no_grad():
        output = model.generate(
            inputs["input_ids"],
            max_length=max_length,
            temperature=0.7,
            top_p=0.9,
            do_sample=True
        )
    
    # Decode and return the generated text
    return tokenizer.decode(output[0], skip_special_tokens=True)

# Test the model
response = generate_text("What is artificial intelligence?")
print(response)

From what I have seen the model format is of meta's base one but I am not sure on how to run it

I am installed the llamma3.2 model from meta directly and got it in this format

-a----         3/10/2025   3:22 PM            209 checklist.chk
-a----         3/10/2025   9:47 AM     6425585114 consolidated.00.pth
-a----         3/10/2025   9:27 AM            220 params.json
-a----         3/10/2025   9:27 AM        2183982 tokenizer.model

And I am trying to run it, but it is giving me this error

Traceback (most recent call last):
  File "C:\Users\user\.llama\checkpoints\Llama3.2-3B-Instruct\LLAMA.PY", line 14, in <module>
    model = LlamaForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 262, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\modeling_utils.py", line 3808, in from_pretrained       
    raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in director

running this code

# Install required packages if you haven't already
# pip install torch transformers

import torch
from transformers import LlamaForCausalLM, LlamaTokenizer

# Path to your model files
model_path = "C:/Users/user/.llama/checkpoints/Llama3.2-3B-Instruct"  # The folder containing the files you listed

# Load the tokenizer
tokenizer = LlamaTokenizer.from_pretrained(model_path)

# Load the model
model = LlamaForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.float16,  # Use float16 for efficiency
    low_cpu_mem_usage=True,     # Helps with memory usage
    device_map="auto"           # Automatically use GPU if available
)

# Generate text
def generate_text(prompt, max_length=100):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    # Generate
    with torch.no_grad():
        output = model.generate(
            inputs["input_ids"],
            max_length=max_length,
            temperature=0.7,
            top_p=0.9,
            do_sample=True
        )
    
    # Decode and return the generated text
    return tokenizer.decode(output[0], skip_special_tokens=True)

# Test the model
response = generate_text("What is artificial intelligence?")
print(response)

From what I have seen the model format is of meta's base one but I am not sure on how to run it

• python
• large-language-model
• llama

Share Improve this question Follow asked Mar 13 at 7:21 JoeJoe 2344 bronze badges 2

• Why not use Ollama to run the model locally? – Ifeanyi Idiaye Commented Mar 13 at 8:50
• I want to configure it manually to do a specific task – Joe Commented Mar 13 at 10:13

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

It seems that your model has not been installed completely. One way can be fetch it from HuggingFace and store it in the place, like the following:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "meta-llama/Llama-3-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

model.save_pretrained("C:/Users/user/.llama/checkpoints/Llama3.2-3B-Instruct")
tokenizer.save_pretrained("C:/Users/user/.llama/checkpoints/Llama3.2-3B-Instruct")

In that way, you will be sure that all files are there.

本文标签： pythonMeta llama 32 3b model local downloadStack Overflow