|
19 | 19 | <table>
|
20 | 20 | <thead>
|
21 | 21 | <tr>
|
22 |
| - <th>GPU Brand</th> |
23 |
| - <th>Windows Support</th> |
24 |
| - <th>Linux Support</th> |
| 22 | + <th>GPU</th> |
| 23 | + <th>Windows</th> |
| 24 | + <th>Linux</th> |
| 25 | + <th>Requirements</th> |
25 | 26 | </tr>
|
26 | 27 | </thead>
|
27 | 28 | <tbody>
|
28 | 29 | <tr>
|
29 | 30 | <td>Nvidia</td>
|
30 |
| - <td>✅ (requires CUDA 11.7 or 11.8)</td> |
31 |
| - <td>✅ (requires CUDA 11.7 or 11.8)</td> |
| 31 | + <td>✅</td> |
| 32 | + <td>✅</td> |
| 33 | + <td>CUDA 11.8 or 11.7</td> |
32 | 34 | </tr>
|
33 | 35 | <tr>
|
34 | 36 | <td>AMD</td>
|
35 | 37 | <td>❌</td>
|
36 |
| - <td>✅ (requires ROCm 5.4.2)</td> |
| 38 | + <td>✅</td> |
| 39 | + <td>ROCm 5.4.2</td> |
37 | 40 | </tr>
|
38 | 41 | <tr>
|
39 | 42 | <td>Apple/Metal</td>
|
40 |
| - <td colspan="2" align="center"> ✅ (requires "Xcode Command Line Tools")</td> |
| 43 | + <td colspan="3" align="center"> ✅ ("Xcode Command Line Tools")</td> |
41 | 44 | </tr>
|
42 | 45 | </tbody>
|
43 | 46 | </table>
|
44 | 47 | </div>
|
45 | 48 |
|
46 |
| - |
47 | 49 | <!-- Table of Contents -->
|
48 | 50 |
|
49 | 51 | <div align="center">
|
|
52 | 54 |
|
53 | 55 | <div align="center">
|
54 | 56 | <a href="#installation">Installation</a> |
|
55 |
| - <a href="#usage-guide">Usage</a> | |
| 57 | + <a href="#usage">Usage</a> | |
56 | 58 | <a href="#contact">Contact Me</a>
|
57 | 59 | </div>
|
58 | 60 |
|
59 | 61 | ## Installation
|
60 | 62 |
|
61 |
| -* **Step 1**: Install the appropriate software if you intend to use GPU-acceleration: |
62 |
| - |
| 63 | +* **Step 1** ➜ Install the appropriate software if you intend to use GPU-acceleration: |
63 | 64 | * **For NVIDIA GPUs** ➜ install [CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive) or [CUDA 11.7](https://developer.nvidia.com/cuda-11-7-0-download-archive) for your specific operating system.
|
64 | 65 | * **For AMD GPUs** ➜ install ROCm version 5.4.2; instructions are [HERE](https://rocmdocs.amd.com/en/latest/deploy/linux/quick_start.html) and [HERE](https://rocmdocs.amd.com/en/latest/deploy/linux/index.html).
|
65 | 66 | * Unfortunately, gpu-accleration will [only work on Linux systems](https://github.com/RadeonOpenCompute/ROCm/blob/develop/docs/rocm.md).
|
66 | 67 | * **For Apple/Metal/MPS** ➜ install [Xcode Command Line Tools](https://www.makeuseof.com/install-xcode-command-line-tools/).
|
67 |
| -* **Step 2**: Download or clone this repository to a directory on your computer. |
68 |
| -* **Step 3**: Open a command prompt from within the directory and create a virtual environment: |
| 68 | +* **Step 2** ➜ Download or clone this repository to a directory on your computer. |
| 69 | +* **Step 3** ➜ Open a command prompt from within the directory and create a virtual environment: |
69 | 70 | ```
|
70 | 71 | python -m venv .
|
71 | 72 | ```
|
72 |
| -* **Step 4**: Activate the virtual environment: |
| 73 | +* **Step 4** ➜ Activate the virtual environment: |
73 | 74 | ```
|
74 | 75 | .\Scripts\activate
|
75 | 76 | ```
|
76 |
| -* **Step 5**: Update [PIP](https://pip.pypa.io/en/stable/index.html): |
| 77 | +* **Step 5** ➜ Update [PIP](https://pip.pypa.io/en/stable/index.html): |
77 | 78 | ```
|
78 | 79 | python -m pip install --upgrade pip
|
79 | 80 | ```
|
80 |
| -* **Step 6**: Install PyTorch with the appropriate "build:" |
81 |
| - |
82 |
| - * **Windows/CUDA 11.8:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
83 |
| - * **Windows/CUDA 11.7:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117``` |
84 |
| - * **Windows/CPU-only:** ```pip install torch torchvision torchaudio``` |
85 |
| - * **Linux/CUDA 11.8:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
86 |
| - * **Linux/CUDA 11.7:** ```pip install torch torchvision torchaudio``` |
87 |
| - * **Linux/ROCm 5.4.2:** ```install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2``` |
88 |
| - * **Linux/CPU-only:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu``` |
89 |
| - * **Apple/Metal/MPS:** ```pip pip install torch torchvision torchaudio``` |
| 81 | +* **Step 6** ➜ Install PyTorch with the appropriate "build:" |
| 82 | + |
| 83 | + * **Windows/CUDA 11.8** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
| 84 | + * **Windows/CUDA 11.7** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117``` |
| 85 | + * **Windows/CPU-only** ➜ ```pip install torch torchvision torchaudio``` |
| 86 | + * **Linux/CUDA 11.8** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
| 87 | + * **Linux/CUDA 11.7** ➜ ```pip install torch torchvision torchaudio``` |
| 88 | + * **Linux/ROCm 5.4.2** ➜ ```install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2``` |
| 89 | + * **Linux/CPU-only** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu``` |
| 90 | + * **Apple/Metal/MPS** ➜ ```pip pip install torch torchvision torchaudio``` |
90 | 91 | * **Metal/MPS** [speedup comparison](https://explosion.ai/blog/metal-performance-shaders) to a 5950x and RTX 3090.
|
91 | 92 |
|
92 |
| -* **Step 7:** Doublecheck that you installed gpu-acceleration properly: |
| 93 | +* **Step 7** ➜ Doublecheck that you installed gpu-acceleration properly: |
93 | 94 | ```
|
94 | 95 | python check_gpu.py
|
95 | 96 | ```
|
96 | 97 |
|
97 |
| -* **Step 8**: Install the dependencies listed in [requirements.txt](https://github.com/MicrosoftDocs/visualstudio-docs/blob/main/docs/python/managing-required-packages-with-requirements-txt.md): |
| 98 | +* **Step 8** ➜ Install the dependencies listed in [requirements.txt](https://github.com/MicrosoftDocs/visualstudio-docs/blob/main/docs/python/managing-required-packages-with-requirements-txt.md): |
98 | 99 | ```
|
99 | 100 | pip install -r requirements.txt
|
100 | 101 | ```
|
101 |
| -* **Step 9**: |
102 |
| - * Lastly, you must install the appropriate version of Git (https://git-scm.com/downloads). |
| 102 | +* **Step 9** ➜ Lastly, you must install the appropriate version of Git (https://git-scm.com/downloads). |
103 | 103 |
|
104 | 104 | [Back to top](#top)
|
105 | 105 |
|
106 |
| -## Usage Guide |
| 106 | +## Usage |
107 | 107 |
|
108 | 108 | * **Step 1**: Open a command prompt in the directory of my scripts, activate the virtual environment, and run:
|
109 | 109 | ```
|
110 | 110 | python gui.py
|
111 | 111 | ```
|
112 |
| -* **Step 2**: Click "Download Embedding Model" and download a model. The GUI will hang. Wait. Proceed to the next step. |
113 |
| -* **Step 3**: Click "Select Embedding Model Directory" and select the DIRECTORY containing the model you want to use. |
114 |
| -* **Step 4**: Click "Choose Documents for Database" and choose one or more PDF files to put in the database. |
| 112 | +* **Step 2** ➜ Click "Download Embedding Model" and download a model. The GUI will hang. Wait. Proceed to the next step. |
| 113 | +* **Step 3** ➜ Click "Select Embedding Model Directory" and select the DIRECTORY containing the model you want to use. |
| 114 | +* **Step 4** ➜ Click "Choose Documents for Database" and choose one or more PDF files to put in the database. |
115 | 115 | * Must have had OCR done on them or have text in them already. If you can select the text when viewing it, you're good.
|
116 |
| -* **Step 5**: Click "Create Vector Database." The GUI will hang. Watch "CUDA" usage. When CUDA drops to zero, proceed to the next step. |
117 |
| -* **Step 6**: Open up LM Studio and load a model (only Llama2-based models currently work). |
118 |
| -* **Step 7**: Click "Start Server" within LM Studio, enter your question, and click "Submit Question." |
| 116 | +* **Step 5** ➜ Click "Create Vector Database." The GUI will hang. Watch "CUDA" usage. When CUDA drops to zero, proceed to the next step. |
| 117 | +* **Step 6** ➜ Open up LM Studio and load a model (only Llama2-based models currently work). |
| 118 | +* **Step 7** ➜ Click "Start Server" within LM Studio, enter your question, and click "Submit Question." |
119 | 119 |
|
120 | 120 |
|
121 | 121 | [Back to top](#top)
|
122 | 122 |
|
123 | 123 | ## Contact
|
124 | 124 |
|
125 |
| -All suggestions (positive and negative) are welcome. I can be reached at "bbc@chintellalaw.com" or feel free to message me on the [LM Studio Discord Server](https://discord.gg/aPQfnNkxGC). |
| 125 | +All suggestions (positive and negative) are welcome. "bbc@chintellalaw.com" or feel free to message me on the [LM Studio Discord Server](https://discord.gg/aPQfnNkxGC). |
126 | 126 |
|
127 | 127 | [Back to top](#top)
|
128 | 128 |
|
|
0 commit comments