Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
VILLASweb-backend-go
VILLASweb-backend-go
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 5
    • Issues 5
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 1
    • Merge Requests 1
  • Requirements
    • Requirements
    • List
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • ACS
    • P
      Public
  • VILLASframework
  • VILLASweb-backend-goVILLASweb-backend-go
  • Issues
  • #50

Closed
Open
Opened Nov 23, 2020 by Steffen Vogel@stvogel🎅🏼Owner

Handle large files

Hi @skolen,

How do we plan to handle larger result files?

Pretty standard simulation easily produce CSVs in the 100MB range and larger. Zipping them will likely help a lot. But I am still a bit uncertain if we can stuff everything in Postgres?

How about using an S3 object store for storing the files?

I stumbled upon this issue when implementing the Python code which fill fetch, extract and load the results.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: acs/public/villas/web-backend-go#50