Issues building Kolla images with recent versions of Python requests

  • Post author:
  • Post category:OpenStack

If you find yourself having issues building Kolla docker container images with errors like this: INFO:kolla.common.utils:Using engine: docker INFO:kolla.common.utils:Found the container image folder at /home/mikal/src/kerbside-patches/venv-stable/2023.1-patches/share/kolla/docker ERROR:kolla.common.utils:Unable to connect to container engine daemon, exiting INFO:kolla.common.utils:Exception caught: Error while fetching server API version: Not supported URL scheme http+docker It is likely that you're running a recent version of requests which has an unresolved issue. Try downgrading requests manually, like this: $ ...venvpath.../bin/pip install requests==2.31.0 Collecting requests==2.31.0 Downloading requests-2.31.0-py3-none-any.whl (62 kB) ...snip... Successfully installed requests-2.31.0 Thanks for this stack overflow post for the pointer.  

Continue ReadingIssues building Kolla images with recent versions of Python requests

Validating a keystone token

  • Post author:
  • Post category:OpenStack

Once again I venture into the lands of poorly documented keystoneauth1 calls. This time, I want to be able to validate if a stored keystone authentication token is valid. Here's the best I could come up with, I'd be interested in others have something better. For this to work, we need a service account to create a keystone client with, and then we can ask that client questions about random other tokens... from keystoneauth1 import exceptions from keystoneauth1.identity import v3 from keystoneauth1 import session from keystoneclient.v3 import client def validate_keystone_token(service_auth, token): """Validate a keystone token. Returns True if the token is valid, False otherwise. """ # We need a keystone client as the service service_session = session.Session(auth=service_auth) service_keystone = client.Client(session=service_session) try: user = service_keystone.tokens.validate(token) except exceptions.http.NotFound: return False # Require that there be an access group with our configured name group = None for g in service_keystone.groups.list(): if g.name == 'mygroup': group = g if not group: return False # Require that the user be in that group try: service_keystone.users.check_in_group(user.user_id, group.id) except exceptions.http.NotFound: return False return True # Authenticate the service user service_auth = v3.Password( auth_url='http://kolla.home.stillhq.com:5000', username='admin', password='...', project_name='admin', user_domain_id='default', project_domain_id='default') # Create a token we can test user_auth =…

Continue ReadingValidating a keystone token

Using the openstacksdk with authentication arguments

  • Post author:
  • Post category:OpenStack

I wanted to authenticate against OpenStack recently, and had a lot of trouble finding documentation about how to authenticate just by passing arguments (as opposed to by using clouds.yaml or environment variables). Now that I have a working incantation, I figure I should write it down so I can find it again. Its also disappointing the OpenStack documentation doesn't appear to cover this particularly well... from keystoneauth1.identity import v3 from keystoneauth1 import session from openstack import connection auth = v3.Password( auth_url='http://kolla.home.stillhq.com:5000', username='admin', password='...', project_name='admin', user_domain_id='default', project_domain_id='default') sess = session.Session(auth=auth) conn = connection.Connection(session=sess) print([x.name for x in conn.list_servers()]) This code will authenticate using the arguments provided, and then list all the servers (instances) visible to that user. You're welcome.

Continue ReadingUsing the openstacksdk with authentication arguments

Turnover of Companies in OpenStack: Prevalence and Rationale

  • Post author:
  • Post category:OpenStack

This paper examines the withdrawal behaviour of corporate contributors to OpenStack, which seems particularly relevant given most contributions in OpenStack are corporately supported, and corporate engagement is declining over time. Its also directly relevant to my own experiences contributing to the project, so seemed like a thing I should read. One interesting aspect of the study is how they define withdrawal from contributions. For each company, they calculate an individual frequency of contribution, and then use that to determine if the company is still making contributions. That is, of a company only ever contributed once a year, we must wait at least a year to know that they have indeed stopped contributing. The paper finds that in more recent OpenStack releases, more companies are leaving contributions than joining. The authors assert that in general engaged developers are now less experienced than previously, which presents risks in terms of developer effectiveness as well as code quality. However, the paper does note that companies with smaller contributions are more likely to disengage than "sustaining companies", however that's largely because there are a huge number of companies contributing only one developer who makes a small number of commits. Unsurprisingly, the paper notes that…

Continue ReadingTurnover of Companies in OpenStack: Prevalence and Rationale

Deciding when to filter out large scale refactorings from code analysis

  • Post author:
  • Post category:OpenStack

I want to be able to see the level of change between OpenStack releases. However, there are a relatively small number of changes with simply huge amounts of delta in them -- they're generally large refactors or the delete which happens when part of a repository is spun out into its own project. I therefore wanted to explore what was a reasonable size for a change in OpenStack so that I could decide what maximum size to filter away as likely to be a refactor. After playing with a couple of approaches, including just randomly picking a number, it seems the logical way to decide is to simply plot a histogram of the various sizes, and then pick a reasonable place on the curve as the cutoff. Due to the large range of values (from zero lines of change to over a million!), I ended up deciding a logarithmic axis was the way to go. For the projects listed in the OpenStack compute starter kit reference set, that produces the following histogram:I feel that filtering out commits over 10,000 lines of delta feels justified based on that graph. For reference, the raw histogram buckets are:

Continue ReadingDeciding when to filter out large scale refactorings from code analysis

A quick summary of OpenStack release tags

I wanted a quick summary of OpenStack git release tags for a talk I am working on, and it turned out to be way more complicated than I expected. I ended up having to compile a table, and then turn that into a code snippet. In case its useful to anyone else, here it is: Or in python form for those so inclined: RELEASE_TAGS = { 'austin': {'all': '2010.1'}, 'bexar': {'all': '2011.1'}, 'cactus': {'all': '2011.2'}, 'diablo': {'all': '2011.3'}, 'essex': {'all': '2012.1.3'}, 'folsom': {'all': '2012.2.4'}, 'grizzly': {'all': '2013.1.5'}, 'havana': {'all': '2013.2.4'}, 'icehouse': {'all': '2014.1.5'}, 'juno': {'all': '2014.2.4'}, 'kilo': {'all': '2015.1.4'}, 'liberty': { 'glance': '11.0.2', 'keystone': '8.1.2', 'neutron': '7.2.0', 'nova': '12.0.6' }, 'mitaka': { 'glance': '12.0.0', 'keystone': '9.3.0', 'neutron': '8.4.0', 'nova': '13.1.4' }, 'newton': { 'glance': '13.0.0', 'keystone': '10.0.3', 'neutron': '9.4.1', 'nova': '14.1.0' }, 'ocata': { 'glance': '14.0.1', 'keystone': '11.0.4', 'neutron': '10.0.7', 'nova': '15.1.5' }, 'pike': { 'glance': '15.0.2', 'keystone': '12.0.3', 'neutron': '11.0.8', 'nova': '16.1.8' }, 'queens': { 'glance': '16.0.1', 'keystone': '13.0.4', 'neutron': '12.1.1', 'nova': '17.0.13' }, 'rocky': { 'glance': '17.0.1', 'keystone': '14.2.0', 'neutron': '13.0.7', 'nova': '18.3.0' }, 'stein': { 'glance': '18.0.1', 'keystone': '15.0.1', 'neutron': '14.4.2', 'nova': '19.3.2' }, 'train': { 'glance': '19.0.4', 'keystone': '16.0.1', 'neutron': '15.3.0', 'nova': '20.4.1' }, 'ussuri': {…

Continue ReadingA quick summary of OpenStack release tags

Rejected talk proposal: Shaken Fist, thought experiments in simpler IaaS clouds

This proposal was submitted for FOSDEM 2021. Given that acceptances were meant to be sent out on 25 December and its basically a week later I think we can assume that its been rejected. I've recently been writing up my rejected proposals, partially because I've put in the effort to write them and they might be useful elsewhere, but also because I think its important to demonstrate that its not unusual for experienced speakers to be rejected from these events. OpenStack today is a complicated beast -- not only does it try to perform well for large clusters, but it also embraces a diverse set of possible implementations from hypervisors, storage, networking, and more. This was a deliberate tactical choice made by the OpenStack community years ago, forming a so called "Big Tent" for vendors to collaborate in to build Open Source cloud options. It made a lot of sense at the time to be honest. However, OpenStack today finds itself constrained by the large number of permutations it must support, ten years of software and backwards compatability legacy, and a decreasing investment from those same vendors that OpenStack courted so actively. Shaken Fist makes a series of simplifying assumptions…

Continue ReadingRejected talk proposal: Shaken Fist, thought experiments in simpler IaaS clouds

Introducing Shaken Fist

The first public commit to what would become OpenStack Nova was made ten years ago today -- at Thu May 27 23:05:26 2010 PDT to be exact. So first off, happy tenth birthday to Nova! A lot has happened in that time -- OpenStack has gone from being two separate Open Source projects to a whole ecosystem, developers have come and gone (and passed away), and OpenStack has weathered the cloud wars of the last decade. OpenStack survived its early growth phase by deliberately offering a "big tent" to the community and associated vendors, with an expansive definition of what should be included. This has resulted in most developers being associated with a corporate sponser, and hence the decrease in the number of developers today as corporate interest wanes -- OpenStack has never been great at attracting or retaining hobbist contributors. My personal involvement with OpenStack started in November 2011, so while I missed the very early days I was around for a lot and made many of the mistakes that I now see in OpenStack. What do I see as mistakes in OpenStack in hindsight? Well, embracing vendors who later lose interest has been painful, and has increased the…

Continue ReadingIntroducing Shaken Fist

What’s missing from the ONAP community — an open design process

I've been thinking a fair bit about ONAP and its future releases recently. This is in the context of trying to implement a system for a client which is based on ONAP. Its really hard though, because its hard to determine how various components of ONAP are intended to work, or interoperate. It took me a while, but I've realised what's missing here... OpenStack has an open design process. If you want to add a new feature to Nova for example, the first step is you need to write down what the feature is intended to do, how it integrates with the rest of Nova, and how people might use it. The target audience for that document is both the Nova development team, but also people who operate OpenStack deployments. ONAP has no equivalent that I can find. So for example, they say that in Casablanca they are going to implement a "AAI Enricher" to ease lookup of data from external systems in their inventory database, but I can't find anywhere where they explain how the integration between arbitrary external systems and ONAP AAI will work. I think ONAP would really benefit from a good hard look at their design processes…

Continue ReadingWhat’s missing from the ONAP community — an open design process

Learning from the mistakes that even big projects make

The following is a blog post version of a talk presented at pyconau 2018. Slides for the presentation can be found here (as Microsoft powerpoint, or as PDF), and a video of the talk (thanks NextDayVideo!) is below:

 

OpenStack is an orchestration system for setting up virtual machines and associated other virtual resources such as networks and storage on clusters of computers. At a high level, OpenStack is just configuring existing facilities of the host operating system — there isn’t really a lot of difference between OpenStack and a room full of system admins frantically resolving tickets requesting virtual machines be setup. The only real difference is scale and predictability.

To do its job, OpenStack needs to be able to manipulate parts of the operating system which are normally reserved for administrative users. This talk is the story of how OpenStack has done that thing over time, what we learnt along the way, and what I’d do differently if I had my time again. Lots of systems need to do these things, so even if you never use OpenStack hopefully there are things to be learnt here.

(more…)

Continue ReadingLearning from the mistakes that even big projects make

End of content

No more pages to load