Commit Graph

150 Commits

Author SHA1 Message Date
OpenDev Sysadmins 3e58a941ac OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:49:20 +00:00
Ian Wienand f204158ff4 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I60eaacc2e33cc97825ca411d12d031207666c1c6
2019-03-24 20:35:59 +00:00
gaofei aee1b062ba Replace Chinese punctuation with English punctuation
Change-Id: I0aacff4ad9be89e4c3958c61bf61110d0c3f6486
2018-01-23 16:40:38 +08:00
Zuul 6c0e4825c6 Merge "Support function package md5" 2018-01-20 03:06:33 +00:00
Lingxian Kong 2fc87a1494 Support function package md5
User can specify md5sum for the package when creating function that
Qinling could check. User can also check by herself when downloading
function package.

Change-Id: Ib3d37cd92bd2ed7018f5a4825a5323b2652948c9
Implements: blueprint qinling-function-package-md5
2018-01-19 22:32:57 +13:00
Lingxian Kong f3ffa45aed Check object size in swift
Change-Id: I6c567767940c4770f3cc79ffe384efdafd588b01
2018-01-19 19:19:01 +13:00
Zuul 85645e60cf Merge "Add minikube gate script as backlog" 2018-01-18 04:56:01 +00:00
Lingxian Kong 72a126ff8f Add kubeadm-dind-cluster gate script as backlog
Change-Id: Ie749acc5c6e7f987085cb03702f6f48ab905a601
2018-01-18 16:50:03 +13:00
Lingxian Kong 924d87dbf7 Add minikube gate script as backlog
Change-Id: If018c30ed04d245e1a2170ad764f22307bdeebf3
2018-01-18 16:49:36 +13:00
Lingxian Kong 4b06c280bb Add administrative operations for some resources
Now, admin user could list/show/delete/update most of qinling
resources, except function deletion. Because function deletion
needs to delete the trust created for qinling service by its
owner, only end user has the authority to delete their trust.

Implements: blueprint qinling-admin-operations
Change-Id: I9ec4df59fbf8ac50c96d9677dd74c54677b307a5
2018-01-18 14:45:11 +13:00
Lingxian Kong 3e35a4b7d5 Support webhook
- Allow user create webhook based on the function.
- Webhook can be invoked without authentication.
- Can not delete function associated with webhook.

Another big change is, we are going to use minikube instead of
kubernetes-aio scripts from openstack-helm project.

Implements: blueprint qinling-function-webhook
Change-Id: I85e0b0f999f0d820bfacca9ac3b9af04e80df0d7
2018-01-13 22:50:47 +13:00
Guoqiang Ding 40b886d5a5 Fix the hard code about source type
Change-Id: I13428367bf4e19a236f8ad25a5052a2af3c1c69f
2018-01-08 01:27:50 +00:00
Tobias Urdin 83611f4508 Remove Mistral references
Remove all Mistral references that was left behind
when the project was created.

Change-Id: I6860e4a7f47600651456b66c4fbe301feef04676
2018-01-06 15:22:33 +01:00
Lingxian Kong aa765e2ae9 Support kubernetes python client 4.0.0
From 4.0.0, kubernetes-incubator/client-python uses multiprocessing
libaray to send request to k8s cluster, which is not supported by
eventlet. This patch introduced the following changes to fix the issue:

- Use cotyledon for engine service rather than oslo.service
- Update global requirments
- Provide separate scripts for api and engine service

References:
[1] https://github.com/eventlet/eventlet/issues/147
[2] https://bugs.launchpad.net/taskflow/+bug/1225275

Change-Id: Ib99565e00eedc72c388e8ebec6b7f1453f77f30f
2018-01-05 18:30:25 +13:00
Lingxian Kong 76f87d59ec Fix job handler
Add unit test as well.

Change-Id: I053f02ae1ad74eb83427b2c4496c5a5d744418d2
2017-12-20 21:18:08 +13:00
Lingxian Kong 789bed3c85 Fix function service expiration handler
Add a unit test as well.

Change-Id: I5eaac5891e2c0aba8c14eef896baeb25c7ea9bc4
2017-12-20 13:41:25 +13:00
Lingxian Kong aa1469da68 Fix docker image function
- Make docker image function work, add functional tests
- Use 'result' instead of 'output' in execution response
- Support string as execution input
- Update python runtime

Partially implements: blueprint qinling-functional-tests
Change-Id: Ie7e59983cfbc6f9e8514438e30a854f372a4c4d7
2017-12-19 23:05:38 +13:00
Zuul 11558af9ad Merge "Fix function service cron job failure" 2017-12-18 21:24:41 +00:00
Lingxian Kong 2d8f7e10b9 Fix function service cron job failure
Change-Id: If0622f776d9df613b9557541b0267ebbf16999af
2017-12-18 23:33:15 +13:00
Lingxian Kong 96eb2a1416 Filter executions by status
Partially implements: blueprint qinling-execution-filter

Change-Id: Ia5ef99559044d5f8cd3803caba417e8def0872c8
2017-12-18 23:26:58 +13:00
Lingxian Kong 072605fa63 Introduce etcd to qinling
- Qinling is using etcd for distributed locking, especially for invoking
  functions simultaneously.
- Get rid of function service and function worker mapping table
- Use etcd to store function service url and workers

Change-Id: I0a6abc98534dc95af5b985b6b6b4c0883263a221
2017-12-18 16:14:51 +13:00
Lingxian Kong b3bedc375d Detach function from underlying orchestrator
This is an admin only operation. which gives admin user a safe way to
clean up the underlying resources allocated for the function.

Change-Id: If98ab5235902516f98be7d485e744cf3ea4cd262
2017-12-14 17:38:02 +13:00
Lingxian Kong e120058fbe Execution operations by admin user
Partially implements: blueprint qinling-admin-operations
Change-Id: Ie2f7572b61b0a88a488f8dcceea35e8d7c689abe
2017-12-13 22:04:38 +13:00
Lingxian Kong 939ba95cc4 Support to get workers by admin user
Admin user can get current workers related to a function.

Change-Id: Ia17e52ad1f7541e63b9e815b47751ad94d33e743
2017-12-13 19:45:29 +13:00
Lingxian Kong ddaa5fc471 Admin user should not delete other project's function
Change-Id: If57bd6da55df0e704739fbfb0d904d14069a5073
2017-12-13 14:39:12 +13:00
Zuul 5dc5ec37fb Merge "Support admin user get functions of other projects" 2017-12-13 00:42:35 +00:00
Lingxian Kong 2778c1cc3b Support admin user get functions of other projects
Partially implements: blueprint qinling-admin-operations
Change-Id: I7192c0006fd82e62f751323cc9707479040f5764
2017-12-12 22:32:52 +13:00
Lingxian Kong 20ac516dac Improve Document for Qinling
Partially implements: blueprint qinling-doc
Change-Id: I1c0df8e737400768fa5a239e15c49ea3fbc4dff3
2017-12-12 12:50:14 +13:00
Lingxian Kong 7ba5676ec8 Function execution concurrency improvement
Increase pod based on the execution rate in qinling-engine so it's
qinling's responsibility to decide when and how to autoscale pod
allocation. This behavior is configurable, the admin user can also
scale up the pod via qinling API manually.

Change-Id: Ie0b01481405daf10f495fa9d6389a624a82f0385
Implements: blueprint qingling-autoscaling
2017-12-07 23:15:56 +13:00
Lingxian Kong c19c4dca00 Improve function execution performance
- Improve/Simplify python runtime
- Use non-locking for communication with db when invoking function.
- Add config option to run job handler.
- Pre-fetch runtime when query function in db.

Change-Id: I9e791aecf03d5bf64a39548e6c9bad1b678882af
2017-12-07 10:49:12 +13:00
Guoqiang Ding 01be0d2a1f Fix the deprecated usage of "get_transport"
As log says, 'oslo_messaging.transport.get_transport()' is deprecated.
The reference link of oslo_messaging is at [1].

[1] https://review.openstack.org/#/c/454194/

Change-Id: Ib4e8cc7b30a7541221d9ae2d244577b2dae5518d
2017-12-06 10:59:05 +08:00
Zuul ffaab41c06 Merge "Limit resource consumption of function" 2017-12-04 11:03:51 +00:00
Lingxian Kong 09f5e6dc60 Limit resource consumption of function
Usages of following resources are limited(for at least python runtime):
- cpu
- process number
- file descriptor
- file size

Change-Id: I3bbd9e7a46a970eb0d9e99d1258b7b27407c0d90
Implements: blueprint qinling-container-resource-limitation
2017-12-04 23:26:13 +13:00
lijunbo 0a2fb8be54 Fix releasenotes job is failure
Closes-bug: #1736051
Change-Id: I7026f9811f9c4f1e0254d17f7aaef579932f0669
2017-12-04 12:41:21 +08:00
Lingxian Kong 6c013ddb7b Limit cpu resource consumption of function
Partially implements: blueprint qinling-container-resource-limitation

Change-Id: I2bb850e2d9bcf33e69d562abdbf88a226e1f4b92
2017-11-30 15:10:37 +13:00
Lingxian Kong cfd63da7b5 Limit function memory consumption
- Limit memory utilization of function execution.
- Timeout handling(3 min by default)
- Python runtime improvement

Partially implements: blueprint qinling-container-resource-limitation

Change-Id: I4ea991035a446acf78f8d12991a616f8637f95ec
2017-11-29 19:54:28 +13:00
Lingxian Kong c22dac0df1 Fix function package update
Change-Id: I534978de7ccc65f00ca09e32eddc1b829af47925
2017-11-27 12:04:21 +13:00
Zuul 3720db0ed6 Merge "Enable some off-by-default checks" 2017-11-26 02:21:34 +00:00
Zuul 1602b8d7f4 Merge "Switch from ostestr to stestr" 2017-11-24 01:58:18 +00:00
Zuul 87fee6a936 Merge "Support to update function code" 2017-11-24 01:47:02 +00:00
junboli 6ec05cc210 Switch from ostestr to stestr
This change updates the tox.ini file to call stestr
directly instead of going through ostestr.
For more background you can read the ML posting:
    https://goo.gl/TFvcxy
So now you won't need `--n` to run single tests, and you
can also supply regex directly without even the `--`.
For example:
  `tox -epy27 $regex`

Change-Id: I2b9b7fbd60b2cc5eea8af01e20cde5714df5027a
2017-11-24 01:26:21 +00:00
Lingxian Kong 9f5b474b6f Support to update function code
Change-Id: If7698350925119140b46cf319ad74f3e063ef0a6
Closes-Bug: #1733477
2017-11-24 14:09:59 +13:00
Zuul 581591b92c Merge "Remove unused variables ‘LOG'" 2017-11-23 02:00:14 +00:00
chenghuiyu 045995b079 Enable some off-by-default checks
Some of available checks are disabled by default,
like:[H106] Don’t put vim configuration in source
files;[H203] Use assertIs(Not)None to check for None;
[H904] Delay string interpolations at logging calls

Change-Id: Ic43e8b77cd6a700f3b71e89d6017f11a742fe87f
2017-11-23 06:55:39 +08:00
chenghuiyu 56616ac535 Remove unused variables ‘LOG'
Perhaps it will be used for debbuging, but now it's unused.

Change-Id: Ie7f5b8461f45a30455c2278b5d67cd392a4f49f8
2017-11-22 16:24:52 +08:00
chenghuiyu d491c0786a Fix new release note in releasenotes
Change-Id: If5ebfa49df7a37e9bb41b4fa5ebe0a03b89758ee
2017-11-21 05:34:26 +08:00
ZhongShengping 585225609e Migrate to Zuulv3
Move qinling gate jobs to its repository.

Change-Id: I2bbe466c8a8adcee30d89fac16f885880badcb8c
2017-11-13 15:50:58 +08:00
Zuul b2d82ee744 Merge "Add python-zaqarclient to python runtime" 2017-11-01 20:49:38 +00:00
Lingxian Kong 1ea8b39051 Add python-zaqarclient to python runtime
Add an example using zaqar.

Change-Id: I130a87d2b1315dca8b8712db5baea31c8eebbd9e
2017-11-02 00:19:10 +13:00
Zuul bc238162fa Merge "Fix the job task failure" 2017-10-24 00:08:50 +00:00