Skip to content
Snippets Groups Projects
Commit af912c20 authored by Luca Schluer's avatar Luca Schluer
Browse files

tar entpackt

parent c0475526
Branches first
No related tags found
No related merge requests found
Expand all Show whitespace changes
Inline Side-by-side
Showing
with 2276 additions and 0 deletions
Task1_spmxv/lab-openmp/.Rprofile 0 → 100644
+ 1
− 0
View file @ af912c20
source("renv/activate.R")
Task1_spmxv/lab-openmp/README.md 0 → 100644
+ 148
− 0
View file @ af912c20
# Software Lab: Parallel Programming for Many-Core Architectures with OpenMP
## Handouts
[![pipeline status](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/badges/master/pipeline.svg)](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/commits/master)
[Kmeans](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/kmeans.tar.gz?job=handouts)
[Sobel](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/sobel.tar.gz?job=handouts)
[Merge-sort](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/merge-sort.tar.gz?job=handouts)
[spmxv](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/spmxv.tar.gz?job=handouts)
[skeleton](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/skeleton.tar.gz?job=handouts)
[Env scripts](https://git-ce.rwth-aachen.de/hpc-lecture/lab-openmp/-/jobs/artifacts/master/file/handouts/lab-openmp.tar.gz?job=handouts)
## TODOs
### Regularly every Year
- Setup thesis compute project
- Apply for compute resources (and if desired for an advanced reservation which is available during in-person meetings)
- Collect the students' user-ids on the cluster
- Add all students to the compute project
- Setup Moodle room
- Schedule automatic uploads of tar archives
- Schedule automatic uploads of slides
### Once
- Add example of a good dev. diary
- Add improved reference solution for `kmeans` which:
- initializes the device(s) before entering the time measurement
- uses hybrid parallelism i.e. multi-threading on the CPU as well
- supports multiple devices
- Check why `-O1` improves performance for `spmxv`
## Creating New Tasks
A template is provided in the `skeleton` task, which can also be used as a starting point for creating a new task.
### Adding a Reference Implementation
- create a directory under `tasks` using all lowercase letters, which uniquely identifies the new task
- directories for reference versions should use the naming scheme `XX_Name`, where `XX` are two digits and `Name` is a unique identifier in camelcase
- each newly created task should have two directories `00_Handout` and `01_Basic` which contain the handout version and a serial implementation of the task respectively
### Scripting Integration
- to allow for reuse of benchmarking scripts the following Makefile targets and conventions should be available:
- `build`, which creates the task's main executable
- `run-small`, which runs the task's main executable utilizing a pre-defined small problem size
- `run-large`, which runs the task's main executable utilizing a pre-defined large problem size
- `clean`, which removes all temporary build files from previous compilations
- `archive`, which creates a tar archive of all files relevant for evaluation i.e. omit files which are shared between all groups
- following a common style for passing the number of threads for a given execution, the environment variable `${NTHREADS}` should be read out to set `${OMP_NUM_THREADS}` accordingly
- integrating a new task into the `parse-benchmark.sh` script is possible by providing a custom result parser as described in [this section](#result-parsing-integration-of-new-tasks)
- integrating a new task into the `parse-competition.sh` script is possible by providing a custom prepare scripts as described in [this section](#competition-integration-of-new-tasks)
## Scripts
### Required Directory Hierarchy
```sh
$ tree .
.
├── .Rprofile
├── renv.lock
├── renv
├── ...
├── scripts
│ ├── benchmark
│ │ └── collect-benchmark.sh
│ ├── parse
│ │ ├── parse-benchmark.sh
│ │ └── result-parsers
│ └── visualize
│ ├── visualize-benchmark.R
│ └── visualize-comparison.R
└── tasks
├── merge-sort
├── ...
└── spmxv
```
### Instructions for Dependency Installation
```sh
# this module has to be loaded before executing any Rscript in this repository
$ module load MATH r-project/4.0.2
$ R
> renv::init()
# press 1 to install the required libraries
```
### Overview
```sh
$ tree scripts
scripts
├── batch # directory for batch scripts which are templates for the students
│ ├── slurm.batch.gpu.sh # --> standard execution on CPU backend nodes with GPUs
│ ├── slurm.batch.knl.sh # --> standard execution on KNL backend nodes
│ ├── slurm.interactive.gpu.sh # --> interactive session on CPU backend nodes with GPUs
│ └── slurm.interactive.knl.sh # --> interactive session on KNL backend nodes
├── benchmark # directory for scripts related to benchmarking individual task versions
│ ├── collect-benchmark.sh # --> main script to collect measurements with multiple thread counts
│ └── setup-environment.sh # --> example script which can be sourced to setup a reproducible shell environment
├── competition # directory for scripts related to collecting measurements of multiple task versions (as done in the final student competition evaluation)
│ ├── collect-competition.sh # --> main script to collect measurements for one fixed execution configuration
│ └── prepare/* # --> per-task scripts to create any task specific files before data collection
├── housekeeping # directory for scripts that help manage the software lab
│ ├── make-handout-base.sh # --> script to create a tar archive containing the base directory structure which can be distributed to students
│ ├── make-handout-task.sh # --> script to create tar archives of individual tasks which can be distributed to students
│ └── create-reference-competition.sh # --> script to create tar archives for the reference implementations of a task
├── parse # directory for scripts related to parsing collected measurements into a csv format
│ ├── parse-benchmark.sh # --> main script to summarize an individual benchmark measurement
│ ├── parse-competition.sh # --> main script to summarize a competition measurement
│ └── result-parsers/* # --> per-task scripts to parse the task output into a csv format
└── visualize # directory for scripts related to visualizing measurements from csv results
├── visualize-benchmark.R # --> script to inspect an individual benchmark measurement
├── visualize-comparison.R # --> script to compare multiple benchmark measurements between each other
└── visualize-competition.R # --> script to evaluate a competition measurement
```
### Result Parsing Integration of new Tasks
The generic implementation of the `parse-benchmark.sh` script utilizes scripts under `result-parsers` to extract task specific fields from result files.
Integrating with this process without modifying the source of `parse-benchmark.sh` directly is possible by following these steps:
- create a shell script under `result-parsers` using `${task}.sh` as the script name
- the provided shell script needs to output the task specific csv header fields, when no input file is given (`[ ${#} -eq 0 ]`)
- otherwise the shell script needs to parse the provided input file and output the extracted performance metrics using a csv style
An example is provided as `skeleton.sh`, which can also be used as a starting point for writing your own result parser.
### Competition Integration of new Tasks
The generic implementation of the `collect-competition.sh` script utilizes scripts under `prepare` to prepare task directories considering task specific files.
Integrating with this process without modifying the source of `collect-competition.sh` directly is possible by following these steps:
- create a shell script under `prepare` using `${task}.sh` as the script name
- the shell script needs to create / copy all task specific files, which are not part of the extracted archive, in / into the given directory provided as script argument `${1}`
An example is provided as `skeleton.sh`, which can also be used as a starting point for writing your own prepare script.
Task1_spmxv/lab-openmp/renv.lock 0 → 100644
+ 782
− 0
View file @ af912c20
{
"R": {
"Version": "4.0.2",
"Repositories": [
{
"Name": "CRAN",
"URL": "https://cloud.r-project.org"
}
]
},
"Packages": {
"Formula": {
"Package": "Formula",
"Version": "1.2-4",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "cc8c8c4d61346cde1ca60030ff9c241f",
"Requirements": []
},
"Hmisc": {
"Package": "Hmisc",
"Version": "4.7-1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "498335c7ca1381ae0c07fe226e0f5dde",
"Requirements": [
"Formula",
"base64enc",
"cluster",
"data.table",
"foreign",
"ggplot2",
"gridExtra",
"gtable",
"htmlTable",
"htmltools",
"lattice",
"latticeExtra",
"nnet",
"rpart",
"survival",
"viridis"
]
},
"KernSmooth": {
"Package": "KernSmooth",
"Version": "2.23-17",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "bbff70c8c0357b5b88238c83f680fcd3",
"Requirements": []
},
"MASS": {
"Package": "MASS",
"Version": "7.3-51.6",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "1dad32ac9dbd8057167b2979fb932ff7",
"Requirements": []
},
"Matrix": {
"Package": "Matrix",
"Version": "1.2-18",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "08588806cba69f04797dab50627428ed",
"Requirements": [
"lattice"
]
},
"R6": {
"Package": "R6",
"Version": "2.5.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "470851b6d5d0ac559e9d01bb352b4021",
"Requirements": []
},
"RColorBrewer": {
"Package": "RColorBrewer",
"Version": "1.1-3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "45f0398006e83a5b10b72a90663d8d8c",
"Requirements": []
},
"Rcpp": {
"Package": "Rcpp",
"Version": "1.0.9",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "e9c08b94391e9f3f97355841229124f2",
"Requirements": []
},
"RcppEigen": {
"Package": "RcppEigen",
"Version": "0.3.3.9.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "4c86baed78388ceb06f88e3e9a1d87f5",
"Requirements": [
"Matrix",
"Rcpp"
]
},
"argparse": {
"Package": "argparse",
"Version": "2.1.6",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "46a8d86acfcbf3af639d6066bc7cc33f",
"Requirements": [
"R6",
"findpython",
"jsonlite"
]
},
"backports": {
"Package": "backports",
"Version": "1.4.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "c39fbec8a30d23e721980b8afb31984c",
"Requirements": []
},
"base64enc": {
"Package": "base64enc",
"Version": "0.1-3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "543776ae6848fde2f48ff3816d0628bc",
"Requirements": []
},
"boot": {
"Package": "boot",
"Version": "1.3-25",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "bd51734a754b6c2baf28b2d1ebc11e91",
"Requirements": []
},
"checkmate": {
"Package": "checkmate",
"Version": "2.1.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "147e4db6909d8814bb30f671b49d7e06",
"Requirements": [
"backports"
]
},
"class": {
"Package": "class",
"Version": "7.3-17",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "9267f5dab59a4ef44229858a142bded1",
"Requirements": [
"MASS"
]
},
"cli": {
"Package": "cli",
"Version": "3.4.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "78003c09d258968a4d28107e779edb10",
"Requirements": []
},
"cluster": {
"Package": "cluster",
"Version": "2.1.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "db63a44aab5aadcb6bf2f129751d129a",
"Requirements": []
},
"codetools": {
"Package": "codetools",
"Version": "0.2-16",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "89cf4b8207269ccf82fbeb6473fd662b",
"Requirements": []
},
"colorspace": {
"Package": "colorspace",
"Version": "2.0-3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "bb4341986bc8b914f0f0acf2e4a3f2f7",
"Requirements": []
},
"data.table": {
"Package": "data.table",
"Version": "1.14.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "36b67b5adf57b292923f5659f5f0c853",
"Requirements": []
},
"deldir": {
"Package": "deldir",
"Version": "1.0-6",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "65a3d4e2a1619bb85ae0fb64628da972",
"Requirements": []
},
"devutils": {
"Package": "devutils",
"Version": "0.1.0",
"Source": "Cellar",
"Hash": "21310124dfcca4a2ca133c5b47900ee6",
"Requirements": [
"rstudioapi"
]
},
"digest": {
"Package": "digest",
"Version": "0.6.29",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "cf6b206a045a684728c3267ef7596190",
"Requirements": []
},
"dplyr": {
"Package": "dplyr",
"Version": "1.0.10",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "539412282059f7f0c07295723d23f987",
"Requirements": [
"R6",
"generics",
"glue",
"lifecycle",
"magrittr",
"pillar",
"rlang",
"tibble",
"tidyselect",
"vctrs"
]
},
"ellipsis": {
"Package": "ellipsis",
"Version": "0.3.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "bb0eec2fe32e88d9e2836c2f73ea2077",
"Requirements": [
"rlang"
]
},
"evaluate": {
"Package": "evaluate",
"Version": "0.16",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "9a3d3c345f8a5648abe61608aaa29518",
"Requirements": []
},
"fansi": {
"Package": "fansi",
"Version": "1.0.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "83a8afdbe71839506baa9f90eebad7ec",
"Requirements": []
},
"farver": {
"Package": "farver",
"Version": "2.1.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "8106d78941f34855c440ddb946b8f7a5",
"Requirements": []
},
"fastmap": {
"Package": "fastmap",
"Version": "1.1.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "77bd60a6157420d4ffa93b27cf6a58b8",
"Requirements": []
},
"findpython": {
"Package": "findpython",
"Version": "1.0.7",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "73cb09f8220dfed388bd89d180f0b922",
"Requirements": []
},
"foreign": {
"Package": "foreign",
"Version": "0.8-80",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "ae1b1e15cc6ccb2bc61c0ac33e86d35f",
"Requirements": []
},
"generics": {
"Package": "generics",
"Version": "0.1.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "15e9634c0fcd294799e9b2e929ed1b86",
"Requirements": []
},
"ggplot2": {
"Package": "ggplot2",
"Version": "3.3.6",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "0fb26d0674c82705c6b701d1a61e02ea",
"Requirements": [
"MASS",
"digest",
"glue",
"gtable",
"isoband",
"mgcv",
"rlang",
"scales",
"tibble",
"withr"
]
},
"ggpresent": {
"Package": "ggpresent",
"Version": "0.3.0",
"Source": "Cellar",
"Hash": "7d112cc750f6086eed84766b1f07a655",
"Requirements": [
"RColorBrewer",
"ggplot2"
]
},
"glue": {
"Package": "glue",
"Version": "1.6.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "4f2596dfb05dac67b9dc558e5c6fba2e",
"Requirements": []
},
"gridExtra": {
"Package": "gridExtra",
"Version": "2.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "7d7f283939f563670a697165b2cf5560",
"Requirements": [
"gtable"
]
},
"gtable": {
"Package": "gtable",
"Version": "0.3.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "36b4265fb818f6a342bed217549cd896",
"Requirements": []
},
"highr": {
"Package": "highr",
"Version": "0.9",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "8eb36c8125038e648e5d111c0d7b2ed4",
"Requirements": [
"xfun"
]
},
"htmlTable": {
"Package": "htmlTable",
"Version": "2.4.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "d5b485330dcfe241b50db157bf898e97",
"Requirements": [
"checkmate",
"htmltools",
"htmlwidgets",
"knitr",
"magrittr",
"rstudioapi",
"stringr"
]
},
"htmltools": {
"Package": "htmltools",
"Version": "0.5.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "6496090a9e00f8354b811d1a2d47b566",
"Requirements": [
"base64enc",
"digest",
"fastmap",
"rlang"
]
},
"htmlwidgets": {
"Package": "htmlwidgets",
"Version": "1.5.4",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "76147821cd3fcd8c4b04e1ef0498e7fb",
"Requirements": [
"htmltools",
"jsonlite",
"yaml"
]
},
"interp": {
"Package": "interp",
"Version": "1.1-3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "593dde779b230c60c4b589fd306bcd3e",
"Requirements": [
"Rcpp",
"RcppEigen",
"deldir"
]
},
"isoband": {
"Package": "isoband",
"Version": "0.2.5",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "7ab57a6de7f48a8dc84910d1eca42883",
"Requirements": []
},
"jpeg": {
"Package": "jpeg",
"Version": "0.1-9",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "441ee36360a57b363f4fa3df0c364630",
"Requirements": []
},
"jsonlite": {
"Package": "jsonlite",
"Version": "1.8.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "d07e729b27b372429d42d24d503613a0",
"Requirements": []
},
"knitr": {
"Package": "knitr",
"Version": "1.40",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "caea8b0f899a0b1738444b9bc47067e7",
"Requirements": [
"evaluate",
"highr",
"stringr",
"xfun",
"yaml"
]
},
"labeling": {
"Package": "labeling",
"Version": "0.4.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "3d5108641f47470611a32d0bdf357a72",
"Requirements": []
},
"lattice": {
"Package": "lattice",
"Version": "0.20-41",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "fbd9285028b0263d76d18c95ae51a53d",
"Requirements": []
},
"latticeExtra": {
"Package": "latticeExtra",
"Version": "0.6-30",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "0857b103442b4657416508733c8c6678",
"Requirements": [
"MASS",
"RColorBrewer",
"interp",
"jpeg",
"lattice",
"png"
]
},
"lifecycle": {
"Package": "lifecycle",
"Version": "1.0.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "25f74670fa7d3277fe3ad8c1712a699f",
"Requirements": [
"glue",
"rlang"
]
},
"magrittr": {
"Package": "magrittr",
"Version": "2.0.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "7ce2733a9826b3aeb1775d56fd305472",
"Requirements": []
},
"mgcv": {
"Package": "mgcv",
"Version": "1.8-31",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "4bb7e0c4f3557583e1e8d3c9ffb8ba5c",
"Requirements": [
"Matrix",
"nlme"
]
},
"munsell": {
"Package": "munsell",
"Version": "0.5.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "6dfe8bf774944bd5595785e3229d8771",
"Requirements": [
"colorspace"
]
},
"nlme": {
"Package": "nlme",
"Version": "3.1-148",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "662f52871983ff3e3ef042c62de126df",
"Requirements": [
"lattice"
]
},
"nnet": {
"Package": "nnet",
"Version": "7.3-14",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "0d87e50e11394a7151a28873637d799a",
"Requirements": []
},
"pillar": {
"Package": "pillar",
"Version": "1.8.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "f2316df30902c81729ae9de95ad5a608",
"Requirements": [
"cli",
"fansi",
"glue",
"lifecycle",
"rlang",
"utf8",
"vctrs"
]
},
"pkgconfig": {
"Package": "pkgconfig",
"Version": "2.0.3",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "01f28d4278f15c76cddbea05899c5d6f",
"Requirements": []
},
"png": {
"Package": "png",
"Version": "0.1-7",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "03b7076c234cb3331288919983326c55",
"Requirements": []
},
"purrr": {
"Package": "purrr",
"Version": "0.3.4",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "97def703420c8ab10d8f0e6c72101e02",
"Requirements": [
"magrittr",
"rlang"
]
},
"renv": {
"Package": "renv",
"Version": "0.15.5",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "6a38294e7d12f5d8e656b08c5bd8ae34",
"Requirements": []
},
"rlang": {
"Package": "rlang",
"Version": "1.0.5",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "971c3d698fc06dabdac6bc4bcda72dc4",
"Requirements": []
},
"rpart": {
"Package": "rpart",
"Version": "4.1-15",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "9787c1fcb680e655d062e7611cadf78e",
"Requirements": []
},
"rstudioapi": {
"Package": "rstudioapi",
"Version": "0.14",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "690bd2acc42a9166ce34845884459320",
"Requirements": []
},
"scales": {
"Package": "scales",
"Version": "1.2.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "906cb23d2f1c5680b8ce439b44c6fa63",
"Requirements": [
"R6",
"RColorBrewer",
"farver",
"labeling",
"lifecycle",
"munsell",
"rlang",
"viridisLite"
]
},
"spatial": {
"Package": "spatial",
"Version": "7.3-12",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "58a02ce0150652b96c044bc67a0df2e5",
"Requirements": []
},
"stringi": {
"Package": "stringi",
"Version": "1.7.8",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "a68b980681bcbc84c7a67003fa796bfb",
"Requirements": []
},
"stringr": {
"Package": "stringr",
"Version": "1.4.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "a66ad12140cd34d4f9dfcc19e84fc2a5",
"Requirements": [
"glue",
"magrittr",
"stringi"
]
},
"survival": {
"Package": "survival",
"Version": "3.1-12",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "1c5bb53bd428f3c42a25b7aeb983d8c7",
"Requirements": [
"Matrix"
]
},
"tibble": {
"Package": "tibble",
"Version": "3.1.8",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "56b6934ef0f8c68225949a8672fe1a8f",
"Requirements": [
"fansi",
"lifecycle",
"magrittr",
"pillar",
"pkgconfig",
"rlang",
"vctrs"
]
},
"tidyselect": {
"Package": "tidyselect",
"Version": "1.1.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "17f6da8cfd7002760a859915ce7eef8f",
"Requirements": [
"ellipsis",
"glue",
"purrr",
"rlang",
"vctrs"
]
},
"utf8": {
"Package": "utf8",
"Version": "1.2.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "c9c462b759a5cc844ae25b5942654d13",
"Requirements": []
},
"vctrs": {
"Package": "vctrs",
"Version": "0.4.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "8b54f22e2a58c4f275479c92ce041a57",
"Requirements": [
"cli",
"glue",
"rlang"
]
},
"viridis": {
"Package": "viridis",
"Version": "0.6.2",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "ee96aee95a7a563e5496f8991e9fde4b",
"Requirements": [
"ggplot2",
"gridExtra",
"viridisLite"
]
},
"viridisLite": {
"Package": "viridisLite",
"Version": "0.4.1",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "62f4b5da3e08d8e5bcba6cac15603f70",
"Requirements": []
},
"withr": {
"Package": "withr",
"Version": "2.5.0",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "c0e49a9760983e81e55cdd9be92e7182",
"Requirements": []
},
"xfun": {
"Package": "xfun",
"Version": "0.33",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "1a666f915cd65072f4ccf5b2888d5d39",
"Requirements": []
},
"yaml": {
"Package": "yaml",
"Version": "2.3.5",
"Source": "Repository",
"Repository": "CRAN",
"Hash": "458bb38374d73bf83b1bb85e353da200",
"Requirements": []
}
}
}
Task1_spmxv/lab-openmp/renv/.gitignore 0 → 100644
+ 4
− 0
View file @ af912c20
library/
lock/
python/
staging/
Task1_spmxv/lab-openmp/renv/activate.R 0 → 100644
+ 942
− 0
View file @ af912c20
This diff is collapsed.
Task1_spmxv/lab-openmp/renv/cellar/devutils_0.1.0.tar.gz 0 → 100644
+ 0
− 0
View file @ af912c20
File added
Task1_spmxv/lab-openmp/renv/cellar/ggpresent_0.3.0.tar.gz 0 → 100644
+ 0
− 0
View file @ af912c20
File added
Task1_spmxv/lab-openmp/renv/settings.dcf 0 → 100644
+ 10
− 0
View file @ af912c20
bioconductor.version:
external.libraries:
ignored.packages:
package.dependency.fields: Imports, Depends, LinkingTo
r.version:
snapshot.type: all
use.cache: TRUE
vcs.ignore.cellar: FALSE
vcs.ignore.library: TRUE
vcs.ignore.local: FALSE
Task1_spmxv/lab-openmp/scripts/batch/slurm.batch.gpu.sh 0 → 100755
+ 33
− 0
View file @ af912c20
#!/usr/bin/env zsh
### Job name
#SBATCH --job-name=SWP_ManyCore_OpenMP
#SBATCH --account=<project-id>
###SBATCH --reservation=<advanced-reservation-id>
### File / path where STDOUT will be written, the %j is the job id
#SBATCH --output=output_%j.txt
### Optional: Send mail when job has finished
###SBATCH --mail-type=END
###SBATCH --mail-user=<email-address>
### Request time
#SBATCH --time=01:00:00
### Set Queue
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=96
#SBATCH --exclusive
#SBATCH --gres=gpu:4
# Set this to the correct path
BASE_DIR=~/Desktop/SWP_ManyCore_OpenMP
TASK=spmxv
# name the benchmark folder
VERSION=openmp
cd "${BASE_DIR}"
./scripts/benchmark/collect-benchmark.sh --task "${TASK}" --version "${VERSION}"
Task1_spmxv/lab-openmp/scripts/batch/slurm.interactive.gpu.sh 0 → 100755
+ 29
− 0
View file @ af912c20
#!/usr/bin/env zsh
### Job name
#SBATCH --job-name=SWP_ManyCore_OpenMP
#SBATCH --account=<project-id>
###SBATCH --reservation=<advanced-reservation-id>
### File / path where STDOUT will be written, the %j is the job id
#SBATCH --output=output_%j.txt
### Optional: Send mail when job has finished
###SBATCH --mail-type=END
###SBATCH --mail-user=<email-address>
### Request time
#SBATCH --time=01:00:00
### Set Queue
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=96
#SBATCH --exclusive
#SBATCH --gres=gpu:4
### wait for dispatch
sleep 7200
### execute 'ssh -Y <allocated-node>' from another shell window
### to get X11 forwarding and execute commands interactively
### when you are finished use 'scancel <job-id>' to free the resources
Task1_spmxv/lab-openmp/scripts/benchmark/collect-benchmark.sh 0 → 100755
+ 99
− 0
View file @ af912c20
#!/usr/bin/env zsh
human_date() {
date +'%Y-%m-%d %H:%M:%S'
}
file_date() {
date +'%Y-%m-%dT%Hh%Mm%Ss'
}
# argument parsing
# defaults for arguments
task=spmxv
version=release
num_executions=23
usage() {
echo "usage: collect-benchmark.sh [-t <task>] [-v <version>] [-n <num-executions>] [-h | --help]" 1>&2
echo "" 1>&2
echo "Collect raw benchmarking results" 1>&2
echo "" 1>&2
echo "script arguments:" 1>&2
echo " -h, --help" 1>&2
echo " show this help message and exit" 1>&2
echo " -t TASK, --task TASK" 1>&2
echo " the unique task identifier for which benchmarking is performed (default: ${task})" 1>&2
echo " -v VERSION, --version VERSION" 1>&2
echo " the unique identifier for the benchmarked code version (default: ${version})" 1>&2
echo " -n NUM_EXECUTIONS, --num-executions NUM_EXECUTIONS" 1>&2
echo " the number of executions for each thread count and problem size (default: ${num_executions})" 1>&2
}
while [ "$1" != "" ]; do
case $1 in
-t | --task)
shift
task=${1}
;;
-v | --version)
shift
version=${1}
;;
-n | --num-executions)
shift
num_executions=${1}
;;
-h | --help)
usage
exit 0
;;
*)
usage
exit 1
;;
esac
shift
done
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")
readonly TASK_DIR=${BASE_DIR}/tasks/${task}
readonly BASE_OUTPUT_DIR=${BASE_DIR}/benchmarking/${task}/$(file_date)-${version}
echo "[$(human_date)] starting benchmark execution for task ${task} version ${version}..."
echo "[$(human_date)] writing output to ${BASE_OUTPUT_DIR}..."
mkdir -p "${BASE_OUTPUT_DIR}"
echo $TASK_DIR
if [ -f "${TASK_DIR}/setup-environment.sh" ]; then
echo "[$(human_date)] setting up the execution environment..."
source "${TASK_DIR}/setup-environment.sh" > "${BASE_OUTPUT_DIR}/software.txt" 2>&1
fi
echo "[$(human_date)] collecting system information..."
{
echo "Hostname: $(hostname)"
lscpu
numactl --hardware
} > "${BASE_OUTPUT_DIR}/hardware.txt" 2>&1
{
lsb_release -d
echo "Kernel Release: $(uname -r)"
} >> "${BASE_OUTPUT_DIR}/operating-system.txt" 2>&1
echo "[$(human_date)] rebuilding executable for task ${task}..."
cd "${TASK_DIR}"
make clean build > "${BASE_OUTPUT_DIR}/build.txt" 2>&1
for run in $(seq "${num_executions}"); do
for num_threads in 1 2 4 8 12 24 48 72 96; do
echo "[$(human_date)] running task ${task} execution ${run} with ${num_threads} threads..."
output_dir=${BASE_OUTPUT_DIR}/${run}/${num_threads}
mkdir -p "${output_dir}"
NTHREADS=${num_threads} make run-small > "${output_dir}/small.txt" 2>&1
NTHREADS=${num_threads} make run-large > "${output_dir}/large.txt" 2>&1
done
done
echo "[$(human_date)] finished benchmark execution..."
Task1_spmxv/lab-openmp/scripts/benchmark/setup-environment.sh 0 → 100644
+ 8
− 0
View file @ af912c20
#!/usr/bin/env zsh
module purge
module load GCCcore/.13.2.0
module load Clang/18.1.2-CUDA-12.3.0
module list
Task1_spmxv/lab-openmp/scripts/competition/collect-competition.sh 0 → 100755
+ 131
− 0
View file @ af912c20
#!/usr/bin/env zsh
human_date() {
date +'%Y-%m-%d %H:%M:%S'
}
file_date() {
date +'%Y-%m-%dT%Hh%Mm%Ss'
}
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")
# argument parsing
# defaults for arguments
task=spmxv
directory=${BASE_DIR}/$(date +'%Y')/competition
num_groups=9
num_executions=23
usage() {
echo "usage: collect-competition.sh [-t <task>] [-d <directory>] [-g <num-groups>] [-n <num-executions>] [-h | --help]" 1>&2
echo "" 1>&2
echo "Collect raw competition results" 1>&2
echo "" 1>&2
echo "script arguments:" 1>&2
echo " -h, --help" 1>&2
echo " show this help message and exit" 1>&2
echo " -t TASK, --task TASK" 1>&2
echo " the unique task identifier for which benchmarking is performed (default: ${task})" 1>&2
echo " -d DIRECTORY, --directory DIRECTORY" 1>&2
echo " the directory which contains the groups tar archives (default: ${directory})" 1>&2
echo " -g NUM_GROUPS, --num-groups NUM_GROUPS" 1>&2
echo " the maximum number of groups participating in the competition (default: ${num_groups})" 1>&2
echo " -n NUM_EXECUTIONS, --num-executions NUM_EXECUTIONS" 1>&2
echo " the number of executions for each thread count and problem size (default: ${num_executions})" 1>&2
}
while [ "$1" != "" ]; do
case $1 in
-t | --task)
shift
task=${1}
;;
-d | --directory)
shift
directory=${1}
;;
-g | --num-groups)
shift
num_groups=${1}
;;
-n | --num-executions)
shift
num_executions=${1}
;;
-h | --help)
usage
exit 0
;;
*)
usage
exit 1
;;
esac
shift
done
readonly SHELL_EXE=$(readlink -f /proc/$$/exe)
readonly PREPARE=$(dirname "$(readlink -f "${0}")")/prepare/${task}.sh
if [ ! -f "${PREPARE}" ]; then
echo "ERROR: cannot collect competition results from task '${task}', because the prepare script '${PREPARE}' is missing" 1>&2
exit 1
fi
directory=$(readlink -f "${directory}")
echo "[$(human_date)] collecting competition results for task ${task}..."
readonly BASE_OUTPUT_DIR=${directory}/results
echo "[$(human_date)] writing output to ${BASE_OUTPUT_DIR}..."
mkdir -p "${BASE_OUTPUT_DIR}"
echo "[$(human_date)] collecting system information..."
{
echo "Hostname: $(hostname)"
lscpu
numactl --hardware
} > "${BASE_OUTPUT_DIR}/hardware.txt" 2>&1
{
lsb_release -d
echo "Kernel Release: $(uname -r)"
} > "${BASE_OUTPUT_DIR}/operating-system.txt" 2>&1
cd "${directory}"
num_threads=64
for group in $(seq "${num_groups}"); do
taskdir=${task}-group-${group}
if [ ! -f "${directory}/${taskdir}.tar.gz" ]; then
echo "[$(human_date)] WARNING: skipping group ${group} because its archive could not be found..."
continue
fi
tar -xzf "${directory}/${taskdir}.tar.gz"
group_output_dir=${BASE_OUTPUT_DIR}/${group}
echo "[$(human_date)] writing output to ${group_output_dir} for group ${group}..."
mkdir -p "${group_output_dir}"
"${SHELL_EXE}" "${PREPARE}" "${taskdir}"
(
cd "${taskdir}"
if [ -f setup-environment.sh ]; then
echo "[$(human_date)] setting up the execution environment for group ${group}..."
source setup-environment.sh > "${group_output_dir}/software.txt" 2>&1
else
echo "[$(human_date)] WARNING: no execution environment given for group ${group}..."
fi
echo "[$(human_date)] building executable for group ${group} task ${task}..."
make clean build > "${group_output_dir}/build.txt" 2>&1
for run in $(seq "${num_executions}"); do
echo "[$(human_date)] running task ${task} for group ${group} execution ${run}..."
output_dir=${group_output_dir}/${run}
mkdir -p "${output_dir}"
NTHREADS=${num_threads} make run-small > "${output_dir}/small.txt" 2>&1
NTHREADS=${num_threads} make run-large > "${output_dir}/large.txt" 2>&1
done
)
rm -rf "${taskdir}"
done
echo "[$(human_date)] finished competition collection..."
Task1_spmxv/lab-openmp/scripts/competition/prepare/kmeans.sh 0 → 100644
+ 7
− 0
View file @ af912c20
#!/usr/bin/env zsh
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")")
taskdir=${1}
cp -r "${BASE_DIR}/tasks/kmeans/input" "${taskdir}"
Task1_spmxv/lab-openmp/scripts/competition/prepare/merge-sort.sh 0 → 100644
+ 1
− 0
View file @ af912c20
#!/usr/bin/env zsh
Task1_spmxv/lab-openmp/scripts/competition/prepare/skeleton.sh 0 → 100644
+ 1
− 0
View file @ af912c20
#!/usr/bin/env zsh
Task1_spmxv/lab-openmp/scripts/competition/prepare/sobel.sh 0 → 100644
+ 8
− 0
View file @ af912c20
#!/usr/bin/env zsh
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")")
taskdir=${1}
cp -r "${BASE_DIR}/tasks/sobel/images" "${taskdir}"
cp -r "${BASE_DIR}/tasks/sobel/utils" "${taskdir}"
Task1_spmxv/lab-openmp/scripts/competition/prepare/spmxv.sh 0 → 100644
+ 8
− 0
View file @ af912c20
#!/usr/bin/env zsh
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")")
taskdir=${1}
cp -r "${BASE_DIR}/tasks/spmxv/input-matrix" "${taskdir}"
cp -r "${BASE_DIR}/tasks/spmxv/utils" "${taskdir}"
Task1_spmxv/lab-openmp/scripts/housekeeping/create-reference-competition.sh 0 → 100755
+ 64
− 0
View file @ af912c20
#!/usr/bin/env zsh
readonly BASE_DIR=$(dirname "$(dirname "$(dirname "$(readlink -f "${0}")")")")
# argument parsing
# defaults for arguments
task=spmxv
directory=${BASE_DIR}/2000/competition
usage() {
echo "usage: create-reference-competition.sh [-t <task>] [-v <version>] [-h | --help]" 1>&2
echo "" 1>&2
echo "Create competition ready tar archives to measure the reference implementations" 1>&2
echo "" 1>&2
echo "script arguments:" 1>&2
echo " -h, --help" 1>&2
echo " show this help message and exit" 1>&2
echo " -t TASK, --task TASK" 1>&2
echo " the unique task identifier for which the tar archives are generated (default: ${task})" 1>&2
echo " -d DIRECTORY, --directory DIRECTORY" 1>&2
echo " the directory in which to place the groups tar archives (default: ${directory})" 1>&2
}
while [ "$1" != "" ]; do
case $1 in
-t | --task)
shift
task=${1}
;;
-h | --help)
usage
exit 0
;;
*)
usage
exit 1
;;
esac
shift
done
readonly MAX_NUM_VERSIONS=99
readonly TASK_DIR=${BASE_DIR}/tasks/${task}
cd "${BASE_DIR}"
mkdir -p "${directory}"
for version in $(seq "${MAX_NUM_VERSIONS}"); do
padded_version=$(printf "%0${#MAX_NUM_VERSIONS}d" "${version}")
if ! find "${TASK_DIR}" -maxdepth 1 -mindepth 1 -type d -name "${padded_version}_*" | grep -q .; then
continue
fi
"${BASE_DIR}/scripts/housekeeping/make-handout-task.sh" -t "${task}" -v "$(basename "${TASK_DIR}/${padded_version}_"*)"
mv "${task}.tar.gz" "${TMPDIR}"
cd "${TMPDIR}"
tar -xzf "${task}.tar.gz"
(
cd "tasks/${task}"
GROUP=${version} make archive
mv "${task}-group-${version}.tar.gz" "${directory}"
)
rm -rf "${task}.tar.gz" "tasks/${task}"
done
Task1_spmxv/lab-openmp/scripts/housekeeping/lab-openmp.tar.gz 0 → 100644
+ 0
− 0
View file @ af912c20
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment