LeCun转赞：在苹果M1/M2芯片上跑LLaMA！130亿参数模型仅需4GB内存

-p 'def open_and_return_content(filename):'

def open_and_return_content(filename):    """    Opens file (returning the content) and performs basic sanity checks    """if os.path.isfile(filename):        with open(filename) as f:            content = f.read()            return contentelse:        print('WARNING: file "{}" does not exist'.format(filename), file=sys.stderr)        return ''def get_file_info(filename, fullpath):    """    Get file information (i.e., permission, owner, group, size)    """

pip3 install --pre torch torchvision --extra-index-url https://download.pytorch.org/whl/nightly/cpu

makeI llama.cpp build info:I UNAME_S:  DarwinI UNAME_P:  armI UNAME_M:  arm64I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -DGGML_USE_ACCELERATEI CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthreadI LDFLAGS:   -framework AccelerateI CC:       Apple clang version 14.0.0 (clang-1400.0.29.202)I CXX:      Apple clang version 14.0.0 (clang-1400.0.29.202)cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -DGGML_USE_ACCELERATE   -c ggml.c -o ggml.oc++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread -c utils.cpp -o utils.oc++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread main.cpp ggml.o utils.o -o main  -framework Accelerate./main -husage: ./main [options]options:  -h, --help            show this help message and exit  -s SEED, --seed SEED  RNG seed (default: -1)    -t N, --threads N     number of threads to use during computation (default: 4)    -p PROMPT, --prompt PROMPT                        prompt to start generation with (default: random)    -n N, --n_predict N   number of tokens to predict (default: 128)    --top_k N             top-k sampling (default: 40)    --top_p N             top-p sampling (default: 0.9)    --temp N              temperature (default: 0.8)    -b N, --batch_size N  batch size for prompt processing (default: 8)    -m FNAME, --model FNAME                        model path (default: models/llama-7B/ggml-model.bin)c++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread quantize.cpp ggml.o utils.o -o quantize  -framework Accelerate

{'dim': 4096, 'multiple_of': 256, 'n_heads': 32, 'n_layers': 32, 'norm_eps': 1e-06, 'vocab_size': 32000}n_parts =  1Processing part  0Processing variable: tok_embeddings.weight with shape:  torch.Size([32000, 4096])  and type:  torch.float16Processing variable: norm.weight with shape:  torch.Size([4096])  and type:  torch.float16  Converting to float32Processing variable: output.weight with shape:  torch.Size([32000, 4096])  and type:  torch.float16Processing variable: layers.0.attention.wq.weight with shape:  torch.Size([4096, 4096])  and type:  torch.float16Processing variable: layers.0.attention.wk.weight with shape:  torch.Size([4096, 4096])  and type:  torch.float16Processing variable: layers.0.attention.wv.weight with shape:  torch.Size([4096, 4096])  and type:  torch.float16Processing variable: layers.0.attention.wo.weight with shape:  torch.Size([4096, 4096])  and type:  torch.float16Processing variable: layers.0.feed_forward.w1.weight with shape:  torch.Size([11008, 4096])  and type:  torch.float16Processing variable: layers.0.feed_forward.w2.weight with shape:  torch.Size([4096, 11008])  and type:  torch.float16Processing variable: layers.0.feed_forward.w3.weight with shape:  torch.Size([11008, 4096])  and type:  torch.float16Processing variable: layers.0.attention_norm.weight with shape:  torch.Size([4096])  and type:  torch.float16...Done. Output file: models/7B/ggml-model-f16.bin, (part  0 )

./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2