|
71 | 71 | transD_GP.splittasks(;nsoundings=3, ncores=23, nchainspersounding=5, ppn=24)</code></pre><p>You will get</p><pre><code class="language-julia hljs">[ Info: will require 1 iterations of 3 soundings in one iteration</code></pre><p>Say for an actual MPI job where you have a thousands of cores for a fixed wing system, say <code>ppn=104</code> and you have 10 nodes available with 1040 cpus. Run this before starting, to see how many iterations it will take to do 800 soundings.</p><pre><code class="language-julia hljs">using HiQGA |
72 | 72 | transD_GP.splittasks(;nsoundings=800, ncores=1039, nchainspersounding=7, ppn=104) |
73 | 73 | [ Info: will require 7 iterations of 129 soundings in one iteration</code></pre><p>You can also violate the stay within a node principle if you have a cluster that does efficient message passing, like this. Say you have an actual <code>ppn=104</code> as in the case of the NCI Sandy Rapids nodes. Then for 12 nodes = 104*12 = 1248 CPUs, we ensure this total number is divisible by the <code>ppn</code> we are providing HiQGA. Say this is 48, a nice number for helicopter systems since 48/(5+1) is an integer. We can fool HiQGA into thinking we are staying within nodes of 48 CPUs as it only uses the <code>ppn</code> value we provide it. We can request 1248 CPUs through <code>PBS</code> and a <code>qsub</code> script but check our input first like so:</p><pre><code class="language-julia hljs">transD_GP.splittasks(;nsoundings=1211, ncores=1247, nchainspersounding=5, ppn=48) |
74 | | -[ Info: will require 6 iterations of 207 soundings in one iteration</code></pre><p><a href="https://github.com/GeoscienceAustralia/HiQGA.jl/blob/a8b258d6cef23be7423c9e8652ea0926af28f448/ASEG_Hobart_Workshop_2024/UDF_probabilistic/submit.sh">Here</a> is an example of a massive job qsub submit script.</p><h3 id="Troubleshooting-MPI-set-up"><a class="docs-heading-anchor" href="#Troubleshooting-MPI-set-up">Troubleshooting MPI set up</a><a id="Troubleshooting-MPI-set-up-1"></a><a class="docs-heading-anchor-permalink" href="#Troubleshooting-MPI-set-up" title="Permalink"></a></h3><p>Some folks have reported that the above MPI install provides error messages to the order of "You are using the system provided MPI", indicating that it is not Intel MPI 2021.10.0 that they are working with. In this case, you should first remove MPI.jl exit Julia, then edit <code>~/.julia/prefs/MPI.toml</code> adding in the following lines</p><pre><code class="nohighlight hljs">path = "/apps/intel-mpi/2021.10.0" |
| 74 | +[ Info: will require 6 iterations of 207 soundings in one iteration</code></pre><p><a href="https://github.com/GeoscienceAustralia/HiQGA.jl/blob/a8b258d6cef23be7423c9e8652ea0926af28f448/ASEG_Hobart_Workshop_2024/UDF_probabilistic/submit.sh">Here</a> is an example of a massive job qsub submit script.</p><h3 id="Troubleshooting-MPI-set-up"><a class="docs-heading-anchor" href="#Troubleshooting-MPI-set-up">Troubleshooting MPI set up</a><a id="Troubleshooting-MPI-set-up-1"></a><a class="docs-heading-anchor-permalink" href="#Troubleshooting-MPI-set-up" title="Permalink"></a></h3><p>Some folks have reported that the above MPI install provides error messages to the order of "You are using the system provided MPI", indicating that it is not Intel MPI 2021.10.0 that they are working with. In this case, you should exit Julia, then edit <code>~/.julia/prefs/MPI.toml</code> adding in the following lines</p><pre><code class="nohighlight hljs">path = "/apps/intel-mpi/2021.10.0" |
75 | 75 | library = "/apps/intel-mpi/2021.10.0/lib/release/libmpi.so" |
76 | 76 | binary = "system"</code></pre><p>then go back to Julia in Pkg mode, making sure the intel-mpi module is loaded in BASH and build again MPI.jl</p><pre><code class="nohighlight hljs">module load intel-mpi/2021.10.0 |
77 | 77 | julia</code></pre><p>Within Julia, you can then do </p><pre><code class="nohighlight hljs"># hit ] to enter Pkg mode |
78 | | -pkg>build MPI</code></pre><p>and this should work.</p><h2 id="For-installing-development-mode-pre-release-versions"><a class="docs-heading-anchor" href="#For-installing-development-mode-pre-release-versions">For installing development mode pre-release versions</a><a id="For-installing-development-mode-pre-release-versions-1"></a><a class="docs-heading-anchor-permalink" href="#For-installing-development-mode-pre-release-versions" title="Permalink"></a></h2><pre><code class="nohighlight hljs">pkg> dev HiQGA</code></pre><p><strong>Make a pull request if you wish to submit your change – we welcome feature additions</strong>. If you want to switch back to the official version from development mode, do</p><pre><code class="nohighlight hljs">pkg> free HiQGA</code></pre><h2 id="References-for-AEM-and-CSEM-physics"><a class="docs-heading-anchor" href="#References-for-AEM-and-CSEM-physics">References for AEM and CSEM physics</a><a id="References-for-AEM-and-CSEM-physics-1"></a><a class="docs-heading-anchor-permalink" href="#References-for-AEM-and-CSEM-physics" title="Permalink"></a></h2><ul><li><a href="https://doi.org/10.1093/gji/ggy255">Blatter, D., Key, K., Ray, A., Foley, N., Tulaczyk, S., & Auken, E. (2018). Trans-dimensional Bayesian inversion of airborne transient EM data from Taylor Glacier, Antarctica. Geophysical Journal International, 214(3)</a></li><li><a href="https://openresearch-repository.anu.edu.au/bitstream/1885/49403/4/02Whole.pdf">Brodie, R. C. (2010). Holistic inversion of airborne electromagnetic data. PhD thesis, Australian National University</a>.</li><li><a href="https://doi.org/10.1111/j.1365-246X.2012.05677.x">Ray, A., & Key, K. (2012). Bayesian inversion of marine CSEM data with a trans-dimensional self parametrizing algorithm. Geophysical Journal International, 191(3), 1135-1151.</a></li></ul></article><nav class="docs-footer"><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="auto">Automatic (OS)</option><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="catppuccin-latte">catppuccin-latte</option><option value="catppuccin-frappe">catppuccin-frappe</option><option value="catppuccin-macchiato">catppuccin-macchiato</option><option value="catppuccin-mocha">catppuccin-mocha</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.9.0 on <span class="colophon-date" title="Thursday 27 March 2025 06:13">Thursday 27 March 2025</span>. Using Julia version 1.9.4.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html> |
| 78 | +pkg>build MPI</code></pre><p>and this should work.</p><h2 id="For-installing-development-mode-pre-release-versions"><a class="docs-heading-anchor" href="#For-installing-development-mode-pre-release-versions">For installing development mode pre-release versions</a><a id="For-installing-development-mode-pre-release-versions-1"></a><a class="docs-heading-anchor-permalink" href="#For-installing-development-mode-pre-release-versions" title="Permalink"></a></h2><pre><code class="nohighlight hljs">pkg> dev HiQGA</code></pre><p><strong>Make a pull request if you wish to submit your change – we welcome feature additions</strong>. If you want to switch back to the official version from development mode, do</p><pre><code class="nohighlight hljs">pkg> free HiQGA</code></pre><h2 id="References-for-AEM-and-CSEM-physics"><a class="docs-heading-anchor" href="#References-for-AEM-and-CSEM-physics">References for AEM and CSEM physics</a><a id="References-for-AEM-and-CSEM-physics-1"></a><a class="docs-heading-anchor-permalink" href="#References-for-AEM-and-CSEM-physics" title="Permalink"></a></h2><ul><li><a href="https://doi.org/10.1093/gji/ggy255">Blatter, D., Key, K., Ray, A., Foley, N., Tulaczyk, S., & Auken, E. (2018). Trans-dimensional Bayesian inversion of airborne transient EM data from Taylor Glacier, Antarctica. Geophysical Journal International, 214(3)</a></li><li><a href="https://openresearch-repository.anu.edu.au/bitstream/1885/49403/4/02Whole.pdf">Brodie, R. C. (2010). Holistic inversion of airborne electromagnetic data. PhD thesis, Australian National University</a>.</li><li><a href="https://doi.org/10.1111/j.1365-246X.2012.05677.x">Ray, A., & Key, K. (2012). Bayesian inversion of marine CSEM data with a trans-dimensional self parametrizing algorithm. Geophysical Journal International, 191(3), 1135-1151.</a></li></ul></article><nav class="docs-footer"><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="auto">Automatic (OS)</option><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="catppuccin-latte">catppuccin-latte</option><option value="catppuccin-frappe">catppuccin-frappe</option><option value="catppuccin-macchiato">catppuccin-macchiato</option><option value="catppuccin-mocha">catppuccin-mocha</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.9.0 on <span class="colophon-date" title="Thursday 27 March 2025 09:19">Thursday 27 March 2025</span>. Using Julia version 1.9.4.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html> |
0 commit comments