Add quantize script for batch quantization

[prusnak]

Alternative to #17 suggested in #17 (comment)

[Jettford]

I wrote up a basic python implementation of the same script for Windows users, would it be worth making a pull request to replace the batch script?

import os
import re
import sys

def print_usage():
    print("Usage: llama-quantize.py 7B|13B|30B|65B [--remove-f16]")
    exit(0)

if not len(sys.argv) > 1:
    print_usage()
    
regex_test = re.compile("^[0-9]{1,2}B$")

if not regex_test.match(sys.argv[1]):
    print_usage()
    
model_directory = f"./models/{sys.argv[1]}/"

if not os.path.exists(model_directory):
    print("Failed to find model directory")
    
    print_usage()
    
for file in os.listdir(model_directory):
    if not "ggml-model-f16.bin" in file:
        continue
        
    file = os.path.join(model_directory, file)
    
    new_name = file.replace("f16", "q4_0")
    
    binary_name = "./quantize"
    
    if sys.platform == "win32":
        binary_name += ".exe"
        
        binary_name = binary_name[2:]
        
    os.system(f"{binary_name} {file} {new_name} 2")
    
    if len(sys.argv) > 2:
        if sys.argv[2] == "--remove-f16":
            os.remove(file)

[tmzncty]

我觉得，直接编译出quantize.exe，然后在CMD中运行就好了。
‘‘‘quantize.exe ggml-model-f16.bin ggml-model-q4.bin 2’’’