Skip to content

Commit bf5fdea

Browse files
MaikMichelmmiMaik Michel
authored
Develop (#27)
* Fix: validate tag existence and calculate next version by version label when setting version to major, minor or patch the next version number will be calculated. Requires at least one version tag! * New: Support translated Apps during deployment * New: All Logs and Artifacts are written to log_path. No changes on apply to depot * New: Enablement occurs explizit only on untouched applictaions * New: Changes to depot are always pushed, when not in Pipeline * New: Input new version for tagging when on main or master and no pipeline is running * Fix: setup won't write log_path to gitignore by default * Updated docs and changelog * New: Modified order of folders during build. The job folder is scanned after dml folders. So the job can be configured by dml statements. * New: Output bash version in logs With that the analyse of errors in bash scripts will become much easier * Aligned log output * New: Remove table_ddl when table itself is new in patch This means that we can now create bundled deployments that have already been installed on test instances, for example. * Fix: Set tag on gate or source branch * Fix: Empty branch-folder when tag is checked out * Fix: Abort when log_path is not defined * Add option --forceddl to switch off checking for new table files This will run table_ddl scripts when matching table is present in patch mode (the old flow) * New: alway execute object hooks wether there are files or not * forceddl flag included in release.sh * remove unneccessary output * use another param when running sed on macos (#26) Co-authored-by: mmi <mmi@opitz-consulting.com> * Add retry to git tag to wait for index.lock to disapear * commit changes if stage != develop or build and current branch is main or master * Add quiet to git output * check in loop a bash expression * Removed commit in release when patch will be applied * just a test script * remove test.sh * Fix: no quiet option in git tag or commit * Add remind_me to check on release * Add env DBFLOW_RELEASE_IS_RUNNUNG to determine if release is calling build * Add some tweaking concerning version tags * Fix: Enabling Apps that were not part of the deployment * check remind me after arguments are validated * updated changelog * updated the docs --------- Co-authored-by: mmi <mmi@opitz-consulting.com> Co-authored-by: Maik Michel <Maik.Michel@bfarm.de>
1 parent 2a5614c commit bf5fdea

15 files changed

+849
-409
lines changed

CHANGELOG.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,30 @@
11
# Changelog
22

3+
## [3.0.0 - 2024-06-30]
4+
- New: When using release.sh you can set version to: major, minor or patch. dbFlow will calcualte the right version number for you
5+
- New: During deployment, translated Applications are se offline too.
6+
- New: Logs will be written to a new Environment var: LOG_PATH. From now on, the depot is not responsible for storing log data. So, when using LOG_PATH inside instance directories, logs should be written to ./${LOG_PATH}. And here (instance directory) it is ok to commit the files as well. This will keep a nice install log history.
7+
- Fix: Setting Applications online occurs only to untouched Applications.
8+
- New: When creating deployments and depot path is a git directory as well, the actual artefact is pushed to the remote too. So you can have a Jenkins or something like that listening to pushed on your depot repository.
9+
- New: The version will be a tag in the target branch and will the tag will be moved to the master/main branch as well.
10+
- New: The jobs folder will be executed after the dml folder and it's childs. So, you can consume data in the job files you have used in dml scripts before.
11+
- New: The version of bash is written to the output.
12+
- New: When a table file is marked as new in the target branch by git, it is executed allthough there could be table_ddl files, which alters the same table. This is new and in important change. So when releasing often to a test or uat branch and later just to a master/main branch, then new tables will be created by running just the create-table-script. Not the 10 changes which were created and deployed during the last sprints. See documentation for more infos.
13+
- New: You have an option to use the old behavior (--forceddl)
14+
- New: Now object-hooks will always be executed. Weather there were objects in the specific file or not.
15+
- New: Environment Var: `REMIND_ME` When this var is set, it will be prompted, when building a deployment, just for you to remind you on something specific.
16+
- Fix: Many small improvements
17+
18+
19+
## [2.6.0 - 2024-02-15]
20+
New: Input new version for tagging when on main or master and no pipeline is running
21+
New: Changes to depot are always pushed, when not in Pipeline
22+
New: Enablement occurs explizit only on untouched applictaions
23+
New: All Logs and Artifacts are written to log_path. No changes on apply to depot
24+
New: Support translated Apps during deployment
25+
New: validate tag existence and calculate next version by version label
26+
Fix: echo out instead of printf when writing changelog
27+
328
## [2.5.0 - 2024-01-23]
429
- New: During patch, scan explicitly for a subfolder
530
inside `tables/tables_ddl` with the name of the target-branch

apply.sh

Lines changed: 189 additions & 92 deletions
Large diffs are not rendered by default.

build.sh

Lines changed: 245 additions & 204 deletions
Large diffs are not rendered by default.

docs/concept.md

Lines changed: 23 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# The Concept
22

3-
**dbFlow** is based on 4 core topics. These topics are encountered again and again during the work with **dbFLow**. One is the adaptation of the git flow related to the work with database changes, the 2 phase deployment, smartFS, which reflects the directory structure and last but not least the depot, a kind of artifactory.
3+
**dbFlow** is based on 4 core topics. These topics are encountered again and again during the work with **dbFlow**. One is the adaptation of the git flow related to the work with database changes, the 2 phase deployment, smartFS, which reflects the directory structure and last but not least the depot, a kind of artifactory.
44

55
## Git Flow to DB
66

@@ -19,7 +19,7 @@ Development mainly takes place on the develop branch. Features that are not incl
1919

2020
![](images/gitflow_02.png)
2121

22-
For each target branch a separate database or database container is expected. The artifacts that **dbFLow** generates can then be installed into the databases/containers.
22+
For each target branch a separate database or database container is expected. The artifacts that **dbFlow** generates can then be installed into the databases/containers.
2323

2424

2525
## 2 Phase deployment
@@ -49,15 +49,16 @@ With this approach it is easy to map the concept of NighlyBuilds with Jenkins or
4949

5050
> SmartFolderStructure
5151
52-
To support this concept, a **dbFlow** project must have a certain directory structure. **dbFLow** expects the following directories at the root level of the project itself.
52+
To support this concept, a **dbFlow** project must have a certain directory structure. **dbFlow** expects the following directories at the root level of the project itself.
5353

5454
| Folder | Description
5555
| --------------- | ---------------------------------------------------
5656
| **apex** | All APEX application will be stored in this folder
5757
| **db** | This folder contains the required database schemas and their objects.
5858
| **rest** | Here the REST services / modules are stored
5959
| **reports** | binary files for templating purpose
60-
| **static** | In this folder the files are stored, which you upload later <br/>with the upload command to the static files of the respective application. (managed by *`dbFlux`*)
60+
| [**static**] | opt. (managed by **dbFlux**) => In this folder the files are stored, which you upload later <br/>with the upload command to the static files of the respective application.
61+
| [**plugin**] | opt. (managed by **dbFlux**) => In this folder the files are stored, which you upload later <br/>with the upload command to the static files of the respective plugin.
6162

6263

6364
### Project Types
@@ -117,7 +118,7 @@ Each schemafolder inside **db** folder except `_setup` is build with the same st
117118
| ...... packages | Packages containing utPLSQL Unittests
118119

119120

120-
All files must be stored as executable SQL scripts. dbFlow uses SQLPlus or SQLcl to import these scripts.
121+
All files must be stored as executable SQL scripts. dbFlow uses SQLPlus or SQLcl to execute these scripts.
121122

122123
!!! Warning
123124

@@ -157,12 +158,12 @@ alter table employees add (
157158
| - employees.sql | create table (..); |
158159

159160

160-
!!! tip
161+
!!! Important
161162

162-
- Changes to tables are always made 2 times. Once for a new installation (**==init==**) and once for an update (**==patch==**).
163-
- Table scripts may only contain the pure tables and comments. Constraints and indexes must be stored in the appropriate directories.
164-
- Files from the table directory are executed if it is a new installation (**==init==**) or if the table is to be newly created in this update (**==patch==**).
165-
- Within the directories the files to be imported are sorted alphabetically, inside pacages or types all files are sorted alphabetically too but grouped in chunke specs, bodies, sql-files directory.
163+
- **Changes** to tables are always made **2 times**. Once for a new installation (**==init==**) and once for an update (**==patch==**).
164+
- Table scripts **==have to==** only contain the pure tables and comments. Constraints and indexes must be stored in the appropriate directories.
165+
- Files from the table directory are executed if it is a new installation (**==init==**) or if the table is to be newly created in this update (**==patch==**) or the table file is marked new in the target branch by git (*New in Version 3*).
166+
- Within the directories the files to be imported are sorted alphabetically, inside packages or types all files are sorted alphabetically too but grouped in chunk specs, bodies, sql-files directory.
166167

167168

168169
#### init
@@ -187,13 +188,13 @@ alter table employees add (
187188
- views
188189
- mviews
189190
- sources/triggers
190-
- jobs
191191
- tests/packages
192192
- ddl
193193
- ddl/init
194194
- dml
195195
- dml/init
196196
- dml/base
197+
- jobs
197198
- .hooks/post
198199

199200
#### patch
@@ -223,7 +224,6 @@ alter table employees add (
223224
- views
224225
- mviews
225226
- sources/triggers
226-
- jobs
227227
- tests/packages
228228
- ddl
229229
- ddl/patch/post_${*branch-name*}
@@ -232,6 +232,7 @@ alter table employees add (
232232
- dml
233233
- dml/base
234234
- dml/patch/post
235+
- jobs
235236
- .hooks/post
236237

237238
### Hooks
@@ -240,7 +241,7 @@ On each level of the main directories there are so called `.hooks` folders. You
240241

241242
!!! Note
242243

243-
Hookscripts outside the respective schema folders **must** contain the corresponding target schema in the name.
244+
Hookscripts outside the respective db schema folders **must** contain the corresponding target schema in the name.
244245

245246
```
246247
├─ .hooks
@@ -426,12 +427,20 @@ Within the folders the order is alphabetical. If dbFLow finds a module file with
426427

427428
## Depot
428429

429-
When you create a deployment, whether **==INIT==** or **==PATCH==**, the deployment artifact is stored in a so-called depot. From this depot, a CI/CD tool, for example, can later fetch the artifact and install it on the target instance. Within the depot directory, the individual deployments are stored in subdirectories that correspond to the Git branch from which they were created. After a successful installation, the patch, including all logs and temporary files, is placed in the success folder. Errors during the deployment lead to an abort and are stored in the failures subfolder.
430+
When you create a deployment, whether **==INIT==** or **==PATCH==**, the deployment artifact is stored in a so-called depot. From this depot, a CI/CD tool, for example, can later fetch the artifact and install it on the target instance. Within the depot directory, the individual deployments are stored in subdirectories that correspond to the Git branch from which they were created.
431+
430432

431433
![](images/depot_and_flow.png)
432434

433435
I recommend to use a separate repository or directory for each stage DB. This has the advantage that the corresponding directories serve their purpose even without Git and possibly access to the development repository. Theoretically, the repository can also be "doubled" to have a target directory at home and a source directory at the customer.
434436

437+
438+
## Logging
439+
440+
After a successful installation, all logs and temporary created artifacs, are placed in a log folder. Errors during the deployment lead to an abort and are stored in the failures subfolder. The location of the log folder itself is configured in apply.env using the var `LOG_PATH` .
441+
442+
> We recommend using the actual instance folder to log to, to ensure greater transparency.
443+
435444
## Big Picture
436445

437446
![](images/comlete_process.png)

0 commit comments

Comments
 (0)