Skip to content
Snippets Groups Projects
Select Git revision
  • main
  • gitkeep
  • UserBasedDockerImage
  • devcontainer
  • exampefiles
  • testing
  • BenediktHeinrichs-patch-1
  • actions
  • 5-change-the-return-type-to-json
  • workingtarget
  • dev protected
  • v0.5.3
  • v0.5.2
  • v0.5.1
  • v0.5.0
  • v0.4.9
  • v0.4.8
  • v0.4.7
  • v0.4.6
  • v0.4.5
  • v0.4.4
  • v0.4.3
  • v0.4.2
  • v0.4.1
  • v0.4.0
25 results

README.md

Blame
  • user avatar
    Benedikt Heinrichs authored
    f6e74177
    History
    Code owners
    Assign users and groups as approvers for specific file changes. Learn more.

    MetadataExtractor

    MetadataExtractor is a web service built on Flask for extracting metadata as RDF triples from various file types. This service utilizes a REST API for receiving files and returning metadata in multiple formats.

    Requirements

    • Python 3.10+
    • Additional dependencies are listed in the installDependencies.sh file.
    • Additional requirements are listed in the requirements.txt file.

    Installation

    To install MetadataExtractor, follow these steps:

    1. Clone the repository:

      git clone https://github.com/BenediktHeinrichs/MetadataExtractor.git
      cd MetadataExtractor
    2. Run the installDependencies.sh script to install required dependencies & Python packages (Linux):

      ./installDependencies.sh
    3. If you have Docker installed, you can build and run the service using the provided Dockerfile.

    Running the Service

    To start the service:

    1. Using Python directly:

      python server.py
    2. Using Docker:

      • Build the Docker image:
        docker build -t metadataextractor .
      • Run the Docker container:
        docker run -p 36541:36541 metadataextractor

    Configuration

    • The service can be configured using the defaultConfigs.py module.
      • This configuration can be overwritten at every metadata extraction request
    • Logging is set up via the setDefaultLogging() function.
    • Environmental variables such as MAX_CONTENT_LENGTH, METADATA_EXTRACTOR_HOST, and METADATA_EXTRACTOR_PORT can be adjusted as needed.

    Version

    Current API version is defined by the __version__ attribute within the MetadataExtractor module.

    Usage

    The service exposes several endpoints:

    POST /

    • This endpoint accepts form-data with a download url or a file along with optional parameters:
      • identifier: A unique identifier for the file.
      • config: Configuration object for extraction settings. (Example value: { "Extractors": { "Text": [ "SummaryExtract" ] } })
      • creation_date: File's creation date.
      • modification_date: File's modification date.
      • url: Download URL of the file.
      • file: The file to be processed.
      • accept: The Accept header has to be set (default is JSON, recommended is Turtle)
    • Returns extracted metadata in the requested format. (JSON, Turtle, RDF/XML, JSON-LD, TriG)

    GET /defaultConfig

    • Returns the default configuration JSON object for the Metadata Extractor.

    GET /version

    • Returns the current version of the Metadata Extractor.

    API Response Models

    The server uses defined response models to structure the JSON response. This includes the MetadataOutput model for the main endpoint and the Version model for the version endpoint.

    Contributing

    Contributions are welcome, check out the Contribution guidelines! Please feel free to submit a pull request.

    Linting & Fixing

    pip install ruff
    ruff --fix .
    ruff format .

    Development

    Open in Gitpod