Skip to content

GitLab

Commit e383aaec authored by Romin's avatar Romin
Browse files

Version 0.5.2

parents
Hide whitespace changes
Inline Side-by-side
.gitignore 0 → 100644
View file @ e383aaec
test/
build/
__pycache__/
token.txt
coscine.egg-info
\ No newline at end of file
CHANGELOG.md 0 → 100644
View file @ e383aaec
# Coscine Python SDK Changelog
## TODO
- Quota management (setting quota for a resource, detailed quota stats)
- sent invite listings, invite operations (revoke, resend)
- ...
## 0.5.2 - 2021-11-22
```diff
+ Several bugfixes
+ Table printout of forms
+ IPython ready
+ Detailed DOCStrings
+ Detailed and interactive documentation
- Removed GitLab Wiki and switched to Jupyter for examples and documentation
- some restructuring
```
## 0.5.0 - 2021-08-16
```diff
+ Revamp of the entire module
+ Type Hinting
+ Native S3-integration with boto3
+ Member management
+ Metadata preset configuration in ResourceForm
+ Various code examples
- DOCStrings
- breaking changes to previous versions
```
## 0.4.0 - 2021-06-02
```diff
+ Added Resource creator (Limitation: No Default Metadata Configuration possible as of now)
+ Moved Documentation to GitLab Wiki
+ Extended Documentation
```
## 0.3.2 - 2021-05-26
```diff
+ Improved DOCstrings
+ Improved Documentation
+ Fixed some issues
+ Added functionality to modify existing metadata with the MetadataForm
+ Added caching
+ Improved visual feedback / logging
```
## 0.3.0 - 2021-05-12
```diff
+ DOCstrings in sourcecode
+ Added Project creator
+ Added verbose mode with progress bars for visual feedback
+ Added colors
+ Fixed Linked Data Resource Handling
+ Adapted to new REST API upload method
+ Added filtering to getter functions like get_projects and get_resources
+ Added dedicated Documentation
+ Added basic Examples
+ Added more error messages
+ Switched to connection pooling for better system resource utilization
- Renamed metadata template to MetadataForm
```
## 0.2.0 - 2021-04-12
```diff
+ Improved Metadata Management
+ Improved package structure
+ Added PyPi Support
+ Added Changelog, revised README
+ Improved Error Handling
- Removed rdflib, pySHACL dependencies
- Removed shacl application profiles
```
## 0.1.0 - 2021-03-31
```diff
+ Initial Release
```
\ No newline at end of file
LICENSE 0 → 100644
View file @ e383aaec
Copyright (c) 2016-2021 RWTH Aachen University
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
README.md 0 → 100644
View file @ e383aaec
# Coscine Python SDK
![Coscine](data/logo.png)
[Coscine](https://coscine.rwth-aachen.de/) is an integration platform for
research data hosted at RWTH Aachen University.
For more information about Coscine visit the official [Landing Page](https://coscine.de/).
## About
This python3.x package provides an SDK for interfacing with Coscine. It is maintained and
developed by the community and should not be regarded as an official service of RWTH Aachen.
You can use the package to integrate Coscine in your python applications and easily:
- create, edit or delete projects and resources
- invite and manage project members
- upload and download files, resources and projects
- manage your metadata
Even if you plan on using the API directly in a programming language of your choice,
you might want to take a look at this module as it provides you with a handy
debug mode which logs all requests and responses made in the communication with coscine.
This makes it simple to understand how certain workflows should be implemented or why
something in your implementation might not work.
## Showcase
Here is a sneak peek of what the module is able to do.
```python
import coscine
token: str = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
client = coscine.Client(token)
project = client.project("My Project")
print(project.name)
print(project.id)
resource = project.resource("My Project Resource")
print(resource.name)
print(resource.pid)
resource.download(path="./datascience/")
files = resource.objects()
for file in files:
print(file.name)
```
## Documentation
Documentation and installation instructions can be found in the
[src/examples](./src/examples) directory. The sourcecode has been thoroughly documented
with python DOCstrings.
## Contact
To report bugs, request features or resolve questions open an issue inside
of the current git repository.
To contribute code and help improve this package fork the repository and
open a merge request.
## License
This project is Open Source Software and licensed
under the terms of the MIT License.
```
The MIT License (MIT)
Copyright © 2018-2021 RWTH Aachen University
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
```
\ No newline at end of file
doc/Makefile 0 → 100644
View file @ e383aaec
py -m pydoc -w coscine
py -m pydoc -w coscine.project
py -m pydoc -w coscine.resource
py -m pydoc -w coscine.exceptions
py -m pydoc -w coscine.static
py -m pydoc -w coscine.progressbar
py -m pydoc -w coscine.form
py -m pydoc -w coscine.client
py -m pydoc -w coscine.metadata
py -m pydoc -w coscine.object
\ No newline at end of file
doc/generate.bat 0 → 100644
View file @ e383aaec
py -m pydoc -w coscine
py -m pydoc -w coscine.project
py -m pydoc -w coscine.resource
py -m pydoc -w coscine.exceptions
py -m pydoc -w coscine.static
py -m pydoc -w coscine.progressbar
py -m pydoc -w coscine.form
py -m pydoc -w coscine.client
py -m pydoc -w coscine.metadata
py -m pydoc -w coscine.object
\ No newline at end of file
doc/generate.sh 0 → 100644
View file @ e383aaec
py -m pydoc -w coscine
py -m pydoc -w coscine.project
py -m pydoc -w coscine.resource
py -m pydoc -w coscine.exceptions
py -m pydoc -w coscine.static
py -m pydoc -w coscine.progressbar
py -m pydoc -w coscine.form
py -m pydoc -w coscine.client
py -m pydoc -w coscine.metadata
py -m pydoc -w coscine.object
\ No newline at end of file
setup.py 0 → 100644
View file @ e383aaec
import setuptools
from src.coscine.version import __version__
with open("README.md", "r", encoding="utf-8") as fh:
long_description = fh.read()
setuptools.setup(
name = "coscine",
version = __version__,
description = "Coscine Python SDK",
long_description = long_description,
long_description_content_type = "text/markdown",
author = "Coscine",
author_email = "coscine@itc.rwth-aachen.de",
license = "MIT License",
packages = setuptools.find_packages(where="src"),
keywords = [
"Coscine", "RWTH Aachen", "Research Data Management"
],
install_requires = [
"requests",
"requests-toolbelt",
"tqdm",
"colorama",
"prettytable"
],
url = "https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk",
project_urls = {
"Issues": "https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk/-/issues",
"Wiki": "https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk/-/wikis/home"
},
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Intended Audience :: Developers"
],
package_dir = {"": "src"},
python_requires = ">=3.6"
)
\ No newline at end of file
src/coscine/__init__.py 0 → 100644
View file @ e383aaec
###############################################################################
# Coscine Python SDK
# Copyright (c) 2018-2021 RWTH Aachen University
# Git: https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk
# Please direct bug reports, feature requests or questions at the URL above
# by opening an issue.
#
# Coscine is an open source project at RWTH Aachen University for
# the management of research data.
# Visit https://www.coscine.de for more information.
#
# Please note that this python module resembles a community effort
# and not an official service that RWTH Aachen provides support for.
# It has been initially brough to life by the team at Coscine but has since
# been turned over to the community around Coscine.
###############################################################################
###############################################################################
"""
Coscine Python SDK
This python3.x package provides an SDK for interfacing with Coscine.
It is maintained and developed by the community and should not be regarded
as an official service of RWTH Aachen.
You can use the package to integrate Coscine in your python applications
and easily:
- create, edit or delete projects and resources
- invite and manage project members
- upload and download files, resources and projects
- manage your metadata
"""
###############################################################################
from .client import Client
from .project import Project, ProjectForm, ProjectMember
from .resource import Resource, ResourceForm
from .object import Object
from .metadata import MetadataForm, MetadataPresetForm
from .exceptions import *
\ No newline at end of file
src/coscine/banner.py 0 → 100644
View file @ e383aaec
###############################################################################
# Coscine Python SDK
# Copyright (c) 2018-2021 RWTH Aachen University
# Git: https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk
# Please direct bug reports, feature requests or questions at the URL above
# by opening an issue.
#
# Coscine is an open source project at RWTH Aachen University for
# the management of research data.
# Visit https://www.coscine.de for more information.
#
# Please note that this python module resembles a community effort
# and not an official service that RWTH Aachen provides support for.
# It has been initially brough to life by the team at Coscine but has since
# been turned over to the community around Coscine.
###############################################################################
###############################################################################
# This file contains a printable banner. It is printed to stdout
# upon initialization of the SDK with verbose mode enabled.
# The banner includes the current package version and title. Otherwise
# it serves no special purpose and can be disregarded for anything other
# than demonstrations.
###############################################################################
from .version import __version__
###############################################################################
BANNER = \
"""
_
(_)
___ ___ ___ ___ _ _ __ ___
/ __/ _ \/ __|/ __| | '_ \ / _ \
| (_| (_) \__ \ (__| | | | | __/
\___\___/|___/\___|_|_| |_|\___|
____________________________________
Coscine Python SDK %s
____________________________________
""" % __version__
###############################################################################
\ No newline at end of file
src/coscine/client.py 0 → 100644
View file @ e383aaec
###############################################################################
# Coscine Python SDK
# Copyright (c) 2018-2021 RWTH Aachen University
# Git: https://git.rwth-aachen.de/coscine/docs/public/coscine-python-sdk
# Please direct bug reports, feature requests or questions at the URL above
# by opening an issue.
#
# Coscine is an open source project at RWTH Aachen University for
# the management of research data.
# Visit https://www.coscine.de for more information.
#
# Please note that this python module resembles a community effort
# and not an official service that RWTH Aachen provides support for.
# It has been initially brough to life by the team at Coscine but has since
# been turned over to the community around Coscine.
###############################################################################
from typing import Iterable
import urllib
import requests
import json
from .static import StaticServer
from .project import Project, ProjectForm
from .version import __version__
from .banner import BANNER
from .exceptions import *
###############################################################################
class Client:
"""
The client class is the backbone and manager of the SDK and mainly
responsible for the communication and exchange of information
with Coscine servers.
Attributes
-----------
lang : str, "en" or "de"
Language preset for input forms and form data.
verbose : bool
Enables/Disables command line output such as log data.
version : str
Contains the current version string of the python SDK.
static : coscine.StaticServer
Retrieves data from Coscine that rarely/never changes.
"""
lang: str
verbose: bool
session: requests.Session
loglevel: Iterable[str]
version: str
static: StaticServer
###############################################################################
def __init__(self, token: str, lang: str = "en", verbose: bool = False,
loglevel: Iterable[str] = ["LOG", "INFO", "WARN", "REQUEST", "DATA"],\
persistent_caching: bool = False) -> None:
"""
The base class of the Coscine Python SDK. When called it
creates a new Coscine client object.
Parameters
----------
token : str
Coscine API access token
lang : str, "en" or "de", default: "en"
Language preset for input form fields.
verbose : bool, default: False
Enables verbose mode - logging/printing information to stdout.
loglevel : str
Enables which messages are printed in verbose mode.
By default all levels are enabled. Possible values:
Any combination of values inside an Iterable:
["LOG", "INFO", "WARN", "REQUEST", "DATA"]
persistent_caching : bool
Enable to store the cache in a file on deinitialization of
the client object. Will attempt to load the cache file
on initialization if enabled. Leads to a significant speed
boost when making static requests right after init, but may
also lead to invalid data when using old cache, in case
the Coscine API changed. Useful for applications with
a short runtime which get run often.
"""
LANG = ("en", "de")
if type(token) is not str:
raise TypeError("Expected argument 'token' as string!")
if lang not in LANG:
raise ValueError("Invalid value for argument 'lang'! "\
"Possible values are %s." % str(LANG))
self.version = __version__
self.lang = lang
self.verbose = verbose
self.loglevel = loglevel
self.static = StaticServer(self, persistent_caching)
self.session = requests.Session()
self.session.headers = {
"Authorization": "Bearer " + token,
"User-Agent": "Coscine Python SDK %s" % self.version
}
self.log(BANNER)
###############################################################################
def log(self, msg: str, level: str = None) -> None:
"""
Prints a message to stdout if verbose mode is enabled.
Parameters
----------
msg : str
Message as a string.
level : str, default: None
Which loglevel to use for printing.
Possible values: "LOG", "INFO", "WARN", "REQUEST", "DATA", None
"""
LOGLEVELS = ["LOG", "INFO", "WARN", "REQUEST", "DATA"]
if not self.verbose:
return
if level is None:
print(msg)
elif self.verbose and level in LOGLEVELS:
print("[%s] %s" % (level, msg))
else:
raise ValueError("Invalid value for argument 'level'! "\
"Possible values are %s." % str(LOGLEVELS))
###############################################################################
def latest_version(self) -> str:
"""
Retrieves the version string of the latest version of this
package hosted on PyPi. Useful for checking whether the currently
used version is outdated and if an update should be performed.
Examples
--------
>>> if client.version != client.latest_version():
>>> print("Module outdated.")
>>> print("Run 'py -m pip install --upgrade coscine'.")
```
Returns
-------
str
The version string of the Coscine Python SDK hosted on PyPi.
"""
URI = "https://pypi.org/pypi/coscine/json"
data = requests.get(URI).json()
version = data["info"]["version"]
return version
###############################################################################
@staticmethod
def uri(api: str, endpoint: str, *args) -> str:
"""
Constructs a URL for performing a request to the Coscine API.
Parameters
----------
api : str
The target Coscine API endpoint.
endpoint : str
The subendpoint of `api`.
*args
Variable number of arguments of type string to append to the URL.
Arguments are automatically seperated by a slash '/' and
special characters are encoded.
Returns
-------
str
Encoded URL for communicating with Coscine servers.
"""
BASE = "https://coscine.rwth-aachen.de/coscine/api/Coscine.Api.%s/%s"
ENDPOINTS = (
"Blob",
"Metadata",
"Organization",
"Project",
"Resources",
"Tree",
"User",
"ActivatedFeatures"
)
if api not in ENDPOINTS:
raise ValueError("Invalid value for argument 'api'. "\
"Possible values are %s." % str(ENDPOINTS))
uri = BASE % (api, endpoint)
for arg in args:
if arg is None:
continue
uri += "/" + urllib.parse.quote(arg, safe="")
return uri
###############################################################################
def _request(self, method: str, uri: str, **kwargs) -> requests.Response:
"""
Performs a HTTP request to the Coscine Servers.
Parameters
----------
method : str
HTTP request method.
uri : str
Coscine URL generated with Client.uri(...).
**kwargs
Additional keyword arguments forwarded to the requests library.
Raises
------
coscine.exceptions.ConnectionError
If the Coscine servers could not be reached.
coscine.exceptions.AuthorizationError
If the Coscine API token is invalid.
coscine.exceptions.ClientError
If the request resulted in an error.
Returns
-------
requests.Response
The response of the Coscine server as a requests.Response object.
"""
self.log("%s %s" % (method, uri), "REQUEST")
if "data" in kwargs and type(kwargs["data"]) is dict:
kwargs["headers"] = {
"Content-Type": "application/json;charset=utf-8"
}
kwargs["data"] = json.dumps(kwargs["data"])
self.log(json.dumps(kwargs["data"], indent=4), "DATA")
try:
response = self.session.request(method, uri, **kwargs)
response.raise_for_status()
return response
except requests.exceptions.ConnectionError:
raise ConnectionError()
except requests.exceptions.RequestException as e:
if e.response.status_code == 401:
raise AuthorizationError("Invalid API token!")
else:
raise ClientError()
###############################################################################
def get(self, uri: str, **kwargs) -> requests.Response:
"""
Performs a GET request to the Coscine API.
Parameters
----------
uri : str
Coscine URL generated with Client.uri(...).
**kwargs
Additional keyword arguments forwarded to the requests library.
Examples
--------
>>> uri = client.uri("Project", "Project")