Feature #338
using the toolkit installer on "slow" (<=10 MBit/s) connections - tar snapshots instead of git?
| Status: | Closed | Start date: | 2014-11-22 | ||
|---|---|---|---|---|---|
| Priority: | Normal | Due date: | |||
| Assignee: |  Jan Moringen | % Done: | 20% | ||
| Category: | build-generator | Spent time: | - | ||
| Target version: | - |
Description
I am trying to set up a distribution at home over a DSL Connection. This takes forever...
What about a policy that we use tar snapshots for all (or at least external) projects files where possible instead of git?
For example opencv/morse etc take ages to download or fail due to timeouts.
(it seems like the opencv git is also extremly slow right now, its syncing with <100kb/s)
The situation how it is right now (checking out during job config, then twice during build) is not really usable to me.
My solution right now is that i switched back to install the (large) projects manually instead of using the toolkit installer which i really do not want to do.
Related issues
History
#1
Updated by Simon Schulz over 9 years ago
- Description updated (diff)
#2
Updated by Simon Schulz over 9 years ago
the build gen could add a timeout for git pulls. i now do this manually:
job configuration, and under git plugin section:
Click "Add"
Click "Advanced clone behaviours"
then i set the checkout timeout to 120#3
Updated by Simon Schulz over 9 years ago
a nice workaround:
open the project settings and replace the git url with /home/sschulz/src/opencv
after you checked out opencv to ~/src/opencv
#4
Updated by Jan Moringen over 8 years ago
- Category set to build-generator
- Status changed from New to In Progress
- Assignee set to Jan Moringen
- % Done changed from 0 to 20
- Clone operations during build have reduced from two to one (kind of generated jobs has been changed matrix project -> "normal" project)
- Incremental download of updates (i.e.
git pull) has been prototypically implemented for the build-generator side and is currently undergoing testing (see #412) - Incremental download for the Jenkins side may be possible in a similar fashion but this needs exploration
I vote against tar-archives as an additional form of distribution/retrieval because of the added complexity.
#5
Updated by Jan Moringen over 8 years ago
- related to Feature #412: multiple cloning of gits - an idea for load reduction for servers + slow connections added
#6
Updated by Martin Wiechmann over 8 years ago
Please consider this as an alternative:
git clone -b <tagName/branchName> --depth 1
From the git-clone manpage:
--branch <name>, -b <name>
Instead of pointing the newly created HEAD to the branch pointed to by the cloned repository’s HEAD, point to <name> branch instead. In a non-bare repository, this is the branch that will be checked out. --branch can also take tags and detaches the HEAD at that commit in the resulting repository.
The option --depth 1 creates a shallowed copy and efficently reduces both remote system load and transfer time.
#7
Updated by Jan Moringen about 8 years ago
- Status changed from In Progress to Closed
Martin Wiechmann wrote:
Please consider this as an alternative:
[...]
From the git-clone manpage:
--branch <name>, -b <name>Instead of pointing the newly created HEAD to the branch pointed to by the cloned repository’s HEAD, point to <name> branch instead. In a non-bare repository, this is the branch that will be checked out. --branch can also take tags and detaches the HEAD at that commit in the resulting repository.
The build-generator already uses this.
The option
--depth 1creates a shallowed copy and efficently reduces both remote system load and transfer time.
This is a configuration option as well.