You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the realm of computer programming, the significance of comprehensive project documentation, including detailed explanations for each Python file, cannot be overstated. Such documentation serves as the cornerstone for understanding, maintaining, and enhancing the codebase. It provides essential context and rationale for the code, making it easier for current and future developers to comprehend the purpose, functionality, and structure of the software. It not only facilitates current and future developers in grasping the project's purpose and structure but also ensures that the project remains accessible and modifiable over time, significantly easing the learning curve for new team members.
14
32
15
33
Traditionally, creating and maintaining software documentation demanded significant human effort and expertise, a challenge for small teams without dedicated personnel. The introduction of Large Language Models (LLMs) like GPT has transformed this, enabling AI to handle much of the documentation process. This shift allows human developers to focus on verification and fine-tuning, greatly reducing the manual burden of documentation.
@@ -76,91 +94,100 @@ If you're looking to contribute or set up a development environment:
76
94
77
95
### Configuring RepoAgent
78
96
79
-
First, configure the OpenAI API parameters in the config.yml file.
80
-
For details on obtaining these, please refer to [OpenAI API](https://beta.openai.com/docs/developer-quickstart/your-api-keys).
81
-
82
-
In the `config.yml` file, configure other parameters like OpenAI API, the destination repository path, document language, and so on:
83
-
84
-
```yaml
85
-
api_keys:
86
-
gpt-3.5-turbo-16k:
87
-
- api_key: sk-XXXX
88
-
base_url: https://example.com/v1/
89
-
api_type: azure
90
-
api_version: XXX
91
-
engine: GPT-35-Turbo-16k
92
-
# you can use any kwargs supported by openai.ChatCompletion here
93
-
- api_key: sk-xxxxx
94
-
organization: org-xxxxxx
95
-
model: gpt-3.5-turbo-16k
96
-
...
97
-
98
-
default_completion_kwargs:
99
-
model: gpt-4-1106
100
-
temperature: 0.2
101
-
request_timeout: 60
102
-
103
-
repo_path: /path/to/your/repo
104
-
project_hierarchy: .project_hierarchy # This is a folder, where we store the project hierarchy and metainfo. This can be shared with your team members.
105
-
Markdown_Docs_folder: Markdown_Docs # The folder in the root directory of your target repository to store the documentation.
106
-
ignore_list: ["ignore_file1.py", "ignore_file2.py", "ignore_directory"] # Ignore some py files or folders that you don't want to generate documentation for by giving relative paths in ignore_list.
107
-
whitelist_path: /path/of/whitelist_path_json #if you provide the whitelist json with the same structure in Metainfo, RepoAgent will only process the given part. This is useful in a very big project, like "higgingface Transformers"
108
-
109
-
language: en # Two-letter language codes (ISO 639-1 codes), e.g. `language: en`for English. Refer to Supported Language for more languages.
110
-
max_thread_count: 10 # We support multiprocessing to speedup the process
111
-
max_document_tokens: 1024 # the maximum number of tokens in a document generated
112
-
log_level: info
97
+
Before configuring specific parameters forRepoAgent, please ensure that the OpenAI API is configured as an environment variablein the command line:
98
+
99
+
```sh
100
+
export OPENAI_API_KEY=YOUR_API_KEY # on Linux/Mac
101
+
set OPENAI_API_KEY=YOUR_API_KEY # on Windows
102
+
$Env:OPENAI_API_KEY = "YOUR_API_KEY" # on Windows (PowerShell)
103
+
```
104
+
105
+
Use `repoagent configure`if you need to modify the running parameters.
106
+
107
+
```sh
108
+
Enter the path to target repository:
109
+
Enter the project hierarchy file name [.project_doc_record]:
110
+
Enter the Markdown documents folder name [markdown_docs]:
111
+
Enter files or directories to ignore, separated by commas []:
112
+
Enter the language (ISO 639 code or language name, e.g., 'en', 'eng', 'English') [Chinese]:
113
+
Enter the maximum number of threads [4]:
114
+
Enter the maximum number of document tokens [1024]:
115
+
Enter the log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) [INFO]:
116
+
Enter the model [gpt-3.5-turbo]:
117
+
Enter the temperature [0.2]:
118
+
Enter the request timeout (seconds) [60.0]:
119
+
Enter the base URL [https://api.openai.com/v1]:
113
120
```
114
121
115
-
### Run RepoAgent
122
+
## Run RepoAgent
116
123
117
124
Enter the root directory of RepoAgent and try the following commandin the terminal:
118
125
```sh
119
-
python -m repo_agent #this command will generate doc, or update docs(pre-commit-hook will automatically call this)
126
+
repoagent run #this command will generate doc, or update docs(pre-commit-hook will automatically call this)
127
+
```
128
+
129
+
The run command supports the following optional flags (if set, will override config defaults):
130
+
131
+
- `-m`, `--model` TEXT: Specifies the model to use for completion. Default: `gpt-3.5-turbo`
132
+
- `-t`, `--temperature` FLOAT: Sets the generation temperature for the model. Lower values make the model more deterministic. Default: `0.2`
133
+
- `-r`, `--request-timeout` INTEGER: Defines the timeout in seconds for the API request. Default: `60`
134
+
- `-b`, `--base-url` TEXT: The base URL for the API calls. Default: `https://api.openai.com/v1`
135
+
- `-tp`, `--target-repo-path` PATH: The file system path to the target repository. Used as the root for documentation generation. Default: `path/to/your/target/repository`
136
+
- `-hp`, `--hierarchy-path` TEXT: The name or path for the project hierarchy file, used to organize documentation structure. Default: `.project_doc_record`
137
+
- `-mdp`, `--markdown-docs-path` TEXT: The folder path where Markdown documentation will be stored or generated. Default: `markdown_docs`
138
+
- `-i`, `--ignore-list` TEXT: A list of files or directories to ignore during documentation generation, separated by commas.
139
+
- `-l`, `--language` TEXT: The ISO 639 code or language name for the documentation. Default: `Chinese`
140
+
- `-ll`, `--log-level` [DEBUG|INFO|WARNING|ERROR|CRITICAL]: Sets the logging level for the application. Default: `INFO`
141
+
142
+
143
+
You can also try the following feature
120
144
121
-
# you can also try the follow feature
122
-
python -m repo_agent clean #this command will remove repoagent-related cache
123
-
python -m repo_agent print #this command will print how repo-agent parse the target repo
124
-
python -m repo_agent diff #this command will check what docs will be updated/generated based on current code change
145
+
```sh
146
+
repoagent clean # Remove repoagent-related cache
147
+
repoagent print-hierarchy # Print how repo-agent parse the target repo
148
+
repoagent diff # Check what docs will be updated/generated based on current code change
125
149
```
126
150
127
151
If it's your first time generating documentation for the target repository, RepoAgent will automatically create a JSON file maintaining the global structure information and a folder named Markdown_Docs in the root directory of the target repository for storing documents.
128
152
129
-
The paths of the global structure information json file and the documentation folder can be configured in `config.yml`.
130
-
131
-
Once you have initially generated the global documentation for the target repository, or if the project you cloned already contains global documentation information, you can then seamlessly and automatically maintain internal project documentation with your team by configuring the **pre-commit hook** in the target repository!
153
+
Once you have initially generated the global documentation for the target repository, or if the project you cloned already contains global documentation information, you can then seamlessly and automatically maintain internal project documentation with your team by configuring the **pre-commit hook** in the target repository!
132
154
133
-
### Configuring the Target Repository
155
+
### Use `pre-commit`
134
156
135
157
RepoAgent currently supports generating documentation for projects, which requires some configuration in the target repository.
136
158
137
159
First, ensure that the target repository is a git repository and has been initialized.
138
-
```
160
+
161
+
```sh
139
162
git init
140
163
```
141
164
Install pre-commit in the target repository to detect changes in the git repository.
142
165
143
-
```
166
+
```sh
144
167
pip install pre-commit
145
168
```
146
169
Create a file named `.pre-commit-config.yaml` in the root directory of the target repository. An example is as follows:
147
170
148
-
```
171
+
```yml
149
172
repos:
150
173
- repo: local
151
174
hooks:
152
175
- id: repo-agent
153
176
name: RepoAgent
154
-
entry: python path/to/your/repo_agent/runner.py
177
+
entry: repoagent
155
178
language: system
179
+
pass_filenames: false # prevent from passing filenames to the hook
156
180
# You can specify the file types that trigger the hook, but currently only python is supported.
157
181
types: [python]
158
182
```
183
+
159
184
For specific configuration methods of hooks, please refer to [pre-commit](https://pre-commit.com/#plugins).
160
185
After configuring the yaml file, execute the following command to install the hook.
161
-
```
186
+
187
+
```sh
162
188
pre-commit install
163
189
```
190
+
164
191
In this way, each git commit will trigger the RepoAgent's hook, automatically detecting changes in the target repository and generating corresponding documents.
165
192
Next, you can make some modifications to the target repository, such as adding a new file to the target repository, or modifying an existing file.
166
193
You just need to follow the normal git workflow: git add, git commit -m "your commit message", git push
@@ -178,25 +205,21 @@ We utilized the default model **gpt-3.5-turbo** to generate documentation for th
178
205
**In the end, you can flexibly adjust the output format, template, and other aspects of the document by customizing the prompt. We are excited about your exploration of a more scientific approach to Automated Technical Writing and your contributions to the community.**
179
206
180
207
### Exploring chat with repo
208
+
181
209
We conceptualize **Chat With Repo** as a unified gateway for these downstream applications, acting as a connector that links RepoAgent to human users and other AI agents. Our future research will focus on adapting the interface to various downstream applications and customizing it to meet their unique characteristics and implementation requirements.
182
210
183
211
Here we demonstrate a preliminary prototype of one of our downstream tasks: Automatic Q&A for Issues and Code Explanation. You can start the server by running the following code.
184
-
```bash
185
-
python -m repo_agent.chat_with_repo
212
+
213
+
```sh
214
+
repoagent chat-with-repo
186
215
```
187
216
188
217
## ✅ Future Work
189
218
190
-
- [x] Identification and maintenance of parent-child relationship hierarchy structure between objects
- [x] **chat with repo** Chat with the repository by giving code and document at the same time
194
-
- [x] Automatically generate better visualizations such as Gitbook
219
+
- [x] Support install and configure via `pip install repoagent`
195
220
- [ ] Generate README.md automatically combining with the global documentation
196
221
- [ ] **Multi-programming-language support** Support more programming languages like Java, C or C++, etc.
197
222
- [ ] Local model support like Llama, chatGLM, Qwen, GLM4, etc.
198
-
- [ ] Support install and configure via `pip install repoagent`
199
-
- [X] Automatically generate Gitbook for better visualization effects
200
223
201
224
## 🥰 Featured Cases
202
225
@@ -206,45 +229,6 @@ Here are featured cases that have adopted RepoAgent.
206
229
- [ChatDev](https://github.com/OpenBMB/ChatDev): Collaborative AI agents for software development.
207
230
- [XAgent](https://github.com/OpenBMB/XAgent): An Autonomous LLM Agent for Complex Task Solving.
208
231
209
-
## 🇺🇳 Supported Language
210
-
211
-
Set the target language with the two-letter language codes (ISO 639-1 codes), Click on the 'Languages List' section below to expand the list of supported languages.
0 commit comments