Commit Graph

38 Commits

Author SHA1 Message Date
Luigi Toscano 5d05a3db2f Remove CentOS6/RHEL6 support
Deprecated in Pike, and removed from diskimage-builder for a while.

Story: 2001996
Task: 19626
Change-Id: I9b4eab56fcbfcb2e97805fba805eafeeafa5d540
2018-05-15 18:22:50 +02:00
Luigi Toscano d987717768 Migration to Storyboard
Sahara projects migrated to storyboard.openstack.org.
Replace the references to Launchpad, including the bugs now
available as stories.
Fix a reference to github.
Change-Id: Iadba69efc1e310b6a19463d3398bf5c6549acd73
2018-03-17 14:40:54 +01:00
Telles Nobrega 05085a81db Adding Spark 2.2.0
Adding newest version of spark.

Change-Id: Ib2894d5d93d3ecfd17e0fb1eba4d687a97406027
2017-12-13 09:55:41 -03:00
Jeremy Freudberg ca8767104b Configurable Spark version for Vanilla image
Instead of hard-coded Spark 1.6.0, allow use of DIB_SPARK_VERSION (and
its corresponding CLI argument `-s`) to specify which version of Spark
should be included on images for Vanilla plugin.

Change-Id: Ia7c7027c9eadfc9d724733a503990ca78e487ee9
2017-07-18 13:00:11 +00:00
Shu Yingya 20126fbde6 Remove some codes of older version builder
Building CDH image under version 5.5.0 is no longer support.
Remove these useless code.
Also, adding the ambari usage info in sahara-image-create
command.

Change-Id: I6fffe25ee9daf651355611be675137babb67e2a8
2017-04-14 11:24:05 +08:00
Vitaliy Levitski 35f7276131 Remove support for MapR 5.0.0 images
Implements: blueprint remove-mapr-500
Change-Id: I6ce676240d053941de6eaafdddd18d3190f353f7
2017-02-01 15:54:44 +00:00
Shu Yingya 604857af49 [DOC] Update support plugin version into doc
Update the plugin version we support(5.7) and will support(5.9) into doc.

Though "DIB_LOCAL_IMAGE" is defined in project "diskimage-builder",
add a specification in docs will be helpful.

Partially-Implements: blueprint cdh-5-9-support
Change-Id: I11ada0c05d426ea483f7c3b7ffa5b247eb77133c
2016-12-13 15:20:18 +00:00
Grigoriy Rozhkov e568680b7a Add support for building MapR 5.2 prebuilt images
Implements: blueprint add-mapr-520-image-building
Depends-On: I450168cb69f4b053a41b99d6daceaf7896a5d7eb
Change-Id: I33271122f81520b1c7902eb331d886b7cba5a626
2016-09-02 15:38:17 +03:00
Telles Nobrega feff7a5b87 Adding ability to create CentOS7 images with MapR
This patch enables the creation of CentOS7 images with MapR plugin.

Change-Id: I915ca1f0f5e890e9b32b64d0d9e31061b7fe5f2c
2016-08-26 14:18:18 -03:00
Vitaly Gridnev ec50d50387 drop vanilla 2.6.0 support from elements
we don't have vanilla 2.6.0 in supported list
in all current branches of sahara. we can
just drop that. if needed, stable/mitaka
branch should be used for building that image.

Change-Id: I81ed8209f2154f112fe7f6718029b84548793380
2016-08-22 17:41:05 +03:00
pratik-gadiya 70040a060b [HDP2.4] Add ambari 2.2.1.0 element
New version of HDP plugin uses this element for installing Ambari
- Default ambari version changed to 2.2.1.0

partially implements bp: hdp-24-support

Change-Id: I91ac4c304ffff3d665b2129845a430672aeba782
2016-07-05 10:47:48 +00:00
Ethan Gafford 08e4221378 Fixing diskimage-create.sh README.rst
Addressing multiple problems in README.rst:
- Outdated support matrix
- Incorrect and incomplete parameter set and options
- Line wraps
- Script name inconsistencies
- Generalized and complete outdatedness

Change-Id: Iefd1bff54bafe11e519bd295795fa3b4569b731f
Closes-bug: 1596715
2016-07-04 15:25:02 +00:00
Vitaly Gridnev 497e4bb6de remove hdp from elements
Change-Id: I7dbd6b003a7f333673004bb29141b06ace604191
blueprint: remove-hdp206
2016-06-27 21:13:38 +00:00
Michael Ionkin 3626fb6429 Added support of Spark 1.6.0
Spark 1.6.0 is available now for deployment.
changes:
* using spark 1.6.0 version as default version for spark plugin
* using hadoop 2.6 for spark 1.6.0 (by setting up
  DIB_CDH_VERSION="5.4")
* we use only HDFS part of CDH for Spark plugin so we don't need to
  create symbolic links for oozie

Depends-on: I9a46a503c7e52d756c7de8c8694dbfc51f80f2be
bp: support-spark-160

Change-Id: Id3c4fb5a0cb1349824972c1be90ea0e0be64cb4b
2016-02-08 12:49:26 +03:00
Evgeny Sikachev 4523111bcf Fix incorrect command in doc
Fix incorrect command for build plain image
closes-bug: #1515942

Change-Id: I57cbca6482df2fd2cf5a28074e9e55c00d81d655
2015-11-13 13:41:05 +03:00
Sergey Reshetnyak 2853c54590 [HDP2] Add ambari element
New version of HDP plugin uses this element for installing Ambari

partially implements bp: hdp-22-support

Change-Id: Icdbd1832fb0f1a64a0a41dd67b840f0d2a38b8b0
2015-08-31 12:28:06 +03:00
Daniele Venzano f376b0f480 Use Cloudera element for Spark HDFS
Update the Spark element to use the existing hadoop-cloudera element for HDFS
for Spark versions > 1.0, instead of the ad-hoc cloudera-cdh one. For Spark 1.0.2,
CDH4 via the old hadoop-cdh element is used, since a precompiled binary for CDH5
is not available.

This change also makes it possible to specify an arbitrary Spark version via the
new -s commandline switch, reducing the amount of code for supporting future
versions of Spark. The defaults for Spark are 1.3.1 and CDH 5.3, a combination
that works well in our deployments.

A small change is needed in the cloudera element: when creating a Spark image,
only the HDFS packages have to be installed.

README files have been updated to clarify that default versions are tested, while
other combinations are not. A reference to the SparkPlugin wiki page was added
to point to a table of supported versions.

Change-Id: Ifc2a0c8729981e1e1df79b556a4c2e6bd1ba893a
Implements: blueprint support-spark-1-3
Depends-On: I8fa482b6d1d6abaa6633aec309a3ba826a8b7ebb
2015-07-10 14:12:37 +00:00
Sergey Reshetnyak ed0564cc88 Use tox env for building images
Changes:
* Use libraries from tox env instead their downloading
* Use pbr for getting package version in sahara-version element

Change-Id: Ifa7128b2d2bc0dbf5b4bc023d5e47540f4c03408
2015-05-05 14:49:08 +03:00
Sergey Reshetnyak 81d2ab1fd1 Don't use sudo for diskimage-create.sh script
Root privileges needed for only installs packages

Change-Id: I7d7395f3a3cb95f595895c169ae530bb3aa10718
2015-03-26 18:16:43 +03:00
Michael McCune 4bf342a48c Adding new update option to diskimage-create.sh
This change removes the global system updates in favor of only updating
the required packages for diskimage creation. Additionally the script
will not automatically install the updates but will warn the user if the
packages are not installed. A switch is provided to allow the script to
install the updates.

Changes
* adding function to detect an installed package
* adding function to check for a list of installed packages
* adding command line switch for updating (-u)
* removing global update commands
* refactoring update logic to fail gracefully or accept switch
* adding -u switch to build-images gate tool
* adding note in documentation about update command line switch

Closes-Bug: 1375843
Change-Id: Iba5e676fa8ef7a852b71acd9e4c19d32f848285a
2015-02-12 10:11:33 -05:00
Artem Osadchiy 61823d408c Image Elements for MapR
Change-Id: I87fde2f5d5b430b935c3261d31b86a7b68ac3244
Implements: blueprint mapr-image-elements
2014-11-28 14:57:34 +02:00
Telles Nobrega 0a9251fa63 Adding Storm as image option
Adding Storm as one of the images that can be created by
diskimagebuilder.

Implements: blueprint insert-storm-as-image-option
Change-Id: If44ee630d090f4e032a3e0bfb71ee363f55eec56
2014-11-07 13:18:02 -03:00
Denis Egorenko 5d7b16b5f3 Fix problem with cloud-init on nova-network
Add datasource modules, which are not included as default
on Ubuntu 14.04.

Change-Id: I645ea17c14960e12bf8649526273e2d7c619c25f
Closes-bug: #1375645
2014-10-10 12:15:43 +04:00
Chad Roberts bbed50ca81 Refactoring IMAGE_TYPE to BASE_IMAGE_OS
Using IMAGE_TYPE to reflect the OS of the base image
conflicts with the way that the diskimage-builder project
uses IMAGE_TYPE (qcow2 vs tar) and was confusing.
We are now using BASE_IMAGE_OS since it properly
reflects what we are using it for and does not conflict
with the diskimage-builder project.

Change-Id: I9e84077c1bc9b8890076838e34a157cb7a3884db
Closes-Bug: #1378364
2014-10-07 10:28:48 -04:00
iberezovskiy 93eb332c0c Add ability to use CentOS local mirror
We could specify ubuntu and fedora local mirror for image build,
now we can use local mirrors for all type of images.

Yum-mirror element was divided in two elements: fedora-mirror
and centos-mirror

Change-Id: Id5016d8a7e982010b6683c038afa21e639ef4ac0
2014-08-07 13:56:36 +04:00
iberezovskiy 76ae53ad42 Building Cloudera hadoop images with Cloudera Manager
* Add element hadoop-cloudera to install Cloudera hadoop packages
* Edit diskimage-create script to build Cloudera images

Partial implements blueprint cdh-plugin

Change-Id: I0368eb12a6c06c8fc900c0f9cca94758437de1df
2014-07-11 15:12:09 +04:00
Michael McCune 98d4ed5ebf Adding options to control DIB repo branch
* Added the -m option
* Added detection of variable DIB_REPO_BRANCH
* Added tag 0.1.17 as default branch of DIB repo
* Added exit error if both DIB_REPO_BRANCH and -m are specified

Implements: blueprint dib-repo-branch-option
Change-Id: I44f0689f7844085c39b5560c65024b68c3424ad8
2014-06-02 11:42:23 -04:00
iberezovskiy 0580bbe7d7 Modify whole java element to support JAVA_TARGET_LOCATION
This patch modifies the common "java" element in order to allow
users to specify the directory that will contain the JDK install
and this support is for all install types now.

Change-Id: I5ba04ba5e22c7f5f5a6455669bbe8509c042a893
2014-05-19 13:32:22 +04:00
Matthew Farrellee 89c44f543c Remove IDH support
Partially implements blueprint: remove-idh-plugin

Change-Id: Iac90c3ec88e47f225500617afc55e98d0a719940
2014-04-11 16:08:51 -04:00
iberezovskiy d822dc4318 Add ability to set DIB_IMAGE_SIZE parameter for CentOS images
Change-Id: Icdc8a92558fc5df84d040172ef97441db7942526
Partial-bug: #1304100
2014-04-09 13:49:07 +04:00
iberezovskiy 827a8ad7df Add ability to use environment variables for image names
Environment variables:
* ubuntu_[vanilla|spark]_hadoop_[1|2]_image_name
* fedora_vanilla_hadoop_[1|2]_image_name
* centos_[vanilla|hdp]_[hadoop_1|hadoop_2|plain]_image_name

Change-Id: I2440be46ec28f7790fcb7d643ee0c15f11d2d8f4
2014-03-13 16:30:26 +04:00
Michael McCune 0e1241d923 Renaming all Savanna references to Sahara
WARNING:
----
Before merging this commit the alias for
http://sahara-files.mirantis.com needs to be in place.

Also before merging this commit the new openstack git project must be
available at https://git.openstack.org/openstack/sahara-image-elements/

NOTE:
----
The file 'elements/hadoop-hdp/source-repository-hadoopswift' contains a
link to the HortonWorks repository that holds the Hadoop Swift rpm, this
link needs to be updated when HortonWorks makes the change.

Implements: blueprint savanna-renaming-image-elements
Change-Id: Icb9a992f8545535af3a111580ce7c9622d754c67
2014-03-13 15:26:47 +04:00
Denis Egorenko e9493f8573 Adds Intel Distribution Hadoop element to savanna-image-elements
Implements: blueprint idh-image-builder

Change-Id: I195a4fcd9be796bd2994f95f128d4e6893cbc254
2014-03-03 14:26:09 +00:00
iberezovskiy cc14bdbb2b Refactoring of diskimage-create script
The script is divided into blocks corresponding version of installed plugin.

Added options for specifying image type and hadoop version. By using option
you can choose image that you want to test instead of creating them all.

Change-Id: I5be5413e978acae26454a1a98825649f7d6498cd
2014-03-03 14:03:10 +04:00
Daniele Venzano 15b81ac7d9 Add options for Spark and Vanilla plugins
Add the '-p' commandline option to the diskimage-builder script to
select which Savanna plugin to target during image generation.
Vanilla and Spark plugins are supported. By default (if no option is
specified) the Vanilla plugin is selected.

Change-Id: I76af023ebd47bb026308b6034cacf01360739be1
2014-02-11 16:44:47 +01:00
iberezovskiy ae28c0fa42 Add possibility to use local repositories
* add parameters 'DIB_REPO_PATH' and 'SIM_REPO_PATH' for specifying pathes
* script tries to use local repository 'savanna-image-elements' as default

Change-Id: I46f3b5573e68e5dfaf5fd98b86eedc90fa108073
2013-12-18 14:02:58 +04:00
iberezovskiy 6b46f637fc Add CentOS support
* As base image we use our own CentOS cloud image with preinstalled cloud-init
* Configuring that image
* Edit hadoop, mysql element for CentOS support and 'diskimage-create' script

Change-Id: If5570f31fd69c7cc809948dd2fa0249d00323f50
2013-12-06 16:04:25 +04:00
IvanBerezovskiy 2769cf9c8f Add diskimage-creating script, elements for mirrors
* Automatic building Fedora and Ubuntu cloud images with all elements from savanna-extra repository
* Using mirrors for updating images

Change-Id: I12632b5cee42b1dbfd79b7b7c3a7b26962ace625
2013-09-05 19:50:43 +04:00