mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Hacks.Mozilla.Org: Designing for performance: A data-informed approach for Quantum development

Mozilla planet - wo, 21/06/2017 - 16:56

When we announced Project Quantum last October, we talked about how users would benefit from our focus on “performance gains…that will be so noticeable that your entire web experience will feel different.”

We shipped the first significant part of this in Firefox 53, and continue to work on the engineering side. Now let’s dive into the performance side and the work we’re doing to ensure that our users will enjoy a faster Web experience.

What makes work on performance so challenging and why is it so important to include the user from the very beginning?

Performance — a contested subject, to say the least!

Awareness of performance as a UX issue often begins with a negative experience – when things get slow or don’t work as expected. In fact, good performance is already a table stake, something that everyone expects from an online product or service. Outstanding performance will very soon become the new baseline point of reference.

The other issue is that there are different perspectives on performance. For users, performance is about their experience and is very often unspecific. For them, perception of good performance can range from “this is amazingly fast” to “SLOW!”, from “WOW!” to “NO!”. For engineers, performance is about numbers and processes. The probes that collect data in the code often measure one specific task in the pipeline. Measuring and tracking capabilities like Garbage Collection (GC) enables engineers to react to regressions in the data quickly, and work on fixing the root causes.

This is why there can be a disconnect between user experience and engineering efforts at mitigation. We measure garbage collection, but it’s often measured without context, such as whether it runs during page load, while the user interacts with a website, or during event queue idle time. Often, GC is within budget, which means that users will hardly perceive it. More generally, specific aspects of what we measure with our probes can be hard to map to the unspecific experience of performance that users have.

Defining technical and perceived performance

To describe an approach for optimizing performance for users, let us start by defining what performance means. For us, there are two sides to performance: technical performance and perceived performance.

Under technical performance, we include the things that we can measure in the browser: how long page elements take to render, how fast we can parse JavaScript or — and that is often more important to understand — how slow certain things are. Technical performance can be measured and the resulting data can be used to investigate performance issues. Technical performance represents the engineer’s viewpoint.

On the other hand, there is the topic of how users experience performance. When users talk about their browser’s performance, they talk about perceived performance or “Quality of Experience” (QoE). Users express QoE in terms of any perceivable, recognized, and nameable characteristic of the product. In the QoE theory, these are called QoE features. We may assume that these characteristics are related to factors in the product that impact technical performance, the QoE factors, but this is not necessarily given.

A promising approach to user-perceived optimization of performance is to identify those factors that have the biggest impact on QoE features and focus on optimizing their technical performance.

Understanding perception

The first step towards optimizing Quantum for perceived performance is to understand how human perception works. We won’t go into details here, but it’s important to know that there are perception thresholds of duration that we can leverage. The most prominent ones for Web interactions were defined by Jacob Nielsen back in the 1990s, and even today, they are informing user-centric performance models like RAIL. Following Nielsen’s thresholds gives a first good estimate about the budget available for certain tasks to be performed by the browser engine.

With our user research team, we are validating and investigating these perceptual thresholds for modern web content. We are running experiments with users, both in the lab and remotely. Of course, this will only happen with users’ consent and everybody will be able to opt in and opt out of these studies at any time. With tools like Shield, we run a set of experiments that allow us to learn about performance and how to improve it for users.

However, knowing the perceptual thresholds and the respective budget is just an important first step. Following, we will go a bit more into detail about how we use a data-informed approach for benchmarking and optimizing performance during the development of our new browser engine.

Three pillars of perceived Web performance

The challenge with optimizing perceived performance of a browser engine is that there are many components involved in bringing data from the network to our screens. All these components may have an impact on the perceived performance and on the underlying perceptual thresholds. However, users don’t know about this structure and the engine. From their point of view, we can define three main pillars for how users perceive performance on the Web: page load, smoothness and responsiveness.

  • Page load: This is what people notice each time when loading a new page. Users care about fast page loads, and we have seen in user research that this is often the way users determine good or bad performance in their browser. Key events defining the perceptual budget during page load are: an immediate response to the user request for a new page, also known as “First Render” or “First non-blank Paint“, and the moment when all important elements are displayed, currently discussed as Hero Element Timing.
  • Smoothness: Scrolling and panning have become challenging activities on modern websites, with infinite scrolling, parallax effects, and dynamic sticky elements. Animations create a better user experience when interacting with the page. Our users want to enjoy a smooth experience for scrolling the web and web animations, be it on social media pages or when shopping for the latest gadget. Often, people nowadays also refer to smoothness as “always 60 fps”.
  • Responsiveness: Beyond scrolling and panning, the other big group of user interactions on websites are mouse, touch, and keyboard inputs. As modern web services create a native-like experience, user expectations for web services are more demanding, based on what they have come to expect for native apps on their laptops and desktop computers. Users have become sensitive to input latency, so we are currently looking at an ideal maximum delay of 100ms.
Targeted optimization for the whole Web

But how do we optimize these three pillars for the whole of the Web? It’s a bigger job than optimizing the performance of a single web service. In building Firefox, we face the challenge of optimizing our browser engine without knowing which pages our users visit or what they do on the Web, due to our commitment to user privacy. This also limits us in collecting data for specific websites or specific user tasks. However, we want to create the best Quality of Experience for as many users and sites as possible.

To start, we decided to focus on the types of content that are currently most popular with Web users. These categories are:

  • Search (e.g.Yahoo Search, Google, Bing)
  • Productivity (e.g. Yahoo Mail, Gmail, Outlook, GSuite)
  • Social (e.g. Facebook, LinkedIn, Twitter, Reddit)
  • Media (e.g. YouTube, Netflix, SoundCloud, Amazon Video)
  • E-commerce (e.g. eBay or Amazon)
  • News & Reference (e.g. NYTimes, BBC, Wikipedia)

Our goal is to learn from this initial set of categories and the most used sites within them and extend our work on improvements to other categories over time. But how do we now match technical to perceived performance and fix technical performance issues to improve the perceived ones?

A data-informed approach to optimizing a browser engine

The goal of our approach here is to take what matters to users and apply that knowledge to achieve technical impact in the engine. With the basics defined above, our iterative approach for optimizing the engine is as follows:

  1. Identification: Based on the set of categories in focus, we specify scenarios for page load, smoothness, and responsiveness that exceed the performance budget and negatively impact perceived performance.
  2. Benchmarks: We define test cases for the identified scenarios so that they become reproducible and quantifiable in our benchmarking testbeds.
  3. Performance profiles: We record and analyze performance profiles to create a detailed view into what’s happening in the browser engine and guide engineers to identify and fix technical root causes.
Identification of scenarios exceeding performance budget

Input for identifying those scenarios come through different sources. They are either informed by results from user research or can be reported through bugs or user feedback. Here are two examples of such a scenario:

  • Scenario: browser startup
  • Category: a special case for page load
  • Performance budget: 1000ms for First Paint and 1500ms for Hero Element
  • Description: Open the browser by clicking the icon > wait for the browser to be fully loaded as maximized window
  • What to measure: First Paint: browser window appears on Desktop, Hero Element: “Search” placeholder in the search box of the content window
  • Scenario: Open chat window on Facebook
  • Category: Responsiveness
  • Performance budget: 150ms
  • Description: Log in to Facebook > Wait for the homepage to be fully loaded > click on a name in the chat panel to open chat window
  • What to measure: time from mouse-click input event to showing the chat window on screen
Benchmarks

We have built different testbeds that allow us to obtain valid and reproducible results, in order to create a baseline for each of the scenarios, and also to be able to track improvements over time. Talos is a python-driven performance testing framework that, among many other tests, has a defined set of tests for browser startup and page load. It’s been recently updated to match the new requirements and measure events closer to user perception like First Paint.

Hasal, on the other hand, focuses on benchmarks around responsiveness and smoothness. It runs a defined set of scripts that perform the defined scenarios (like the “open chat window” scenario above) and extracts the required timing data through analyzing videos captured during the interaction.

Additionally, there is still a lot of non-automated, manual testing involved, especially for first rounds of baselining new scenarios before scripting them for automated testing. Therefore, we use a HDMI capture card and analyze the recorded videos frame-by-frame manually.

All these testbeds give us data about how critical the identified scenarios are in terms of exceeding their respective perceptual budgets. Running benchmarks regularly (once a week or even more often) for critical scenarios like browser startup also tracks improvements over time and provides good direction when improvements have moved the scenario into the perceptual budget.

Performance profiles

Now that we have defined our scenarios and understand how much improvement is required to create good Quality of Experience, the last step is to enable engineers to achieve these improvements. The way that engineers look at performance problems in the browser engine is through performance profiles. Performance profiles are a snapshot of what happens in the browser engine during a specific user task such as one of our defined scenarios.

A performance profile using the Gecko Profiler. The profile shows Gecko’s main thread, four content threads, and the compositor main thread. Below is the call stack.

 

A profile consists of a timeline with tracing markers, different thread timelines and the call tree. The timeline consists of several rows that indicate interesting events in terms of tracing markers (colored segments). With the timeline, you can also zoom in to get more details for marked areas. The thread timelines show a list of profiled threads, like Gecko’s Main Thread, four content process threads (thanks to multi-process), and the main thread of the compositor process, as seen in the profile above. The x-axis is synced to the timeline above, and the y-axis shows the stack depth at a given point in time. Finally, the call tree shows the collected samples within a given timeframe organized by ‘Running Time’.

It requires some experience to be able to read these performance profiles and translate them into actions. However, because they map critical user scenarios directly to technical performance, performance profiles serve as a good tool to improve the browser engine according to what users care about. The challenge here is to identify root causes to improve performance broadly, rather than focus on specific sites and individual bugs. This is also the reason why we focus on categories of pages and not an individual set of initial websites.

For in-depth information about performance profiles, here is an article and a talk from Ehsan Akhgari about performance profiles. We are continuously working on improving the profiler addon which is now written in React/Redux.

Iterative testing and profiling performance

The initial round of baselining and profiling performance for the scenarios above can help us go from identifying user performance issues to fixing those issues in the browser engine. However, only iterative testing and profiling of performance can ensure that patches that land in the code will also lead to the expected benefits in terms of performance budget.

Additionally, iterative benchmarking will also help identify the impact that a patch has on other critical scenarios. Looking across different performance profiles and capturing comparable interactions or page load scenarios actually leads to fixing root causes. By fixing root causes rather than focusing on one-off cases, we anticipate that we will be able to improve QoE and benefit entire categories of websites and activities.

Continuous performance monitoring with Telemetry

Ultimately, we want to go beyond a specific set of web categories and look at the Web as a whole. We also want to go beyond manual testing, as this is expensive and time-consuming. And we want to apply knowledge that we have obtained from our initial data-driven approach and extend it to monitoring performance across our user base through Telemetry.

We recently added probes to our Telemetry system that will help us to track events that matter to the user, in the wild across all websites, like first non-blank paint during page load. Over time, we will extend the set of probes meaningfully. A good first attempt to define and include probes that are closer to what users perceive has been taken by the Google Chrome team and their Progressive Web Metrics.

A visualization of Progressive Web Metrics during page load and page interaction. The upper field shows the user interaction level and critical interactions related to the technical measures.

 

As mentioned in the beginning, for users performance is a table stake, something that they expect. In this article, we have explored: how we capture issues in perceived performance, how we use benchmarks to measure the criticality of performance issues, and how to fix the issue by looking at performance profiles.

Beyond the scope of the current approach to performance, there’s an even more interesting question: Will improved performance lead to more usage of the browser or changes to how users use their browser? Can performance improvements increase user engagement?

But these are topics that still need more research — and, at some point in time, will be the subject for another blog post.

Meanwhile, if you are now interested to follow along on performance improvements and experience the enhanced performance of the Firefox browser, go download and install the latest Firefox Nightly build and see what you think of its QoE.

Categorieën: Mozilla-nl planet

Chris McDonald: Message Broker: Into String

Mozilla planet - di, 20/06/2017 - 23:18

Strings in most native or performance focused languages tend to present a fair amount of complexity and Rust is no exception to this. There are cases where you have a struct that needs to have a name, such as:

struct ServiceHandle { name: String, }

The first ServiceHandle::new() function I would typically write when first learning Rust would look like this:

fn new(name: String) -> ServiceHandle { // other init stuff ServiceHandle { name: name } }

In the real world I generally have a String to pass to ServiceHandle::new(String) so this API works well. But when I’m writing tests for my code, I want to pass hard coded values with type &'static str. In order to do that, I have to call one of the conversion functions.

ServiceHandle::new("listener".into()); ServiceHandle::new("listener".to_owned()); ServiceHandle::new(String::from("listener"));

If I change the signature to something like:

fn new(name: &'a str) -> ServiceHandle { // other init stuff ServiceHandle { name: name.to_owned() } }

Then I have to remember to prefix String with & when passing to the function. Another possibility is to from_string and from_str which is what I started to rely on next. Then I don’t have to remember as much, just use the right one for the right type.

fn from_str(name: &'a str) -> ServiceHandle { // skip init stuff, let from_string do it ServiceHandle::from_string(name.to_owned()) } fn from_string(name: String) -> ServiceHandle { // other init stuff ServiceHandle { name: name } }

This gets me into a better state where it is easy to remember what string type it takes, but it feels clumsy. Rust provides the From and Into traits that types can implement to enable more generic coding. They are as their name implies a way to automatically change types. They are “reflexive” which means that if one of them is implemented the other can use it. For example, impl From<A> for B would allow you to write a function like fn thing<T: Into<B>>(arg: T) which could be called like thing(A {}).

So the next iteration of my ServiceHandle::new() went generic:

fn new<S: Into<String>>(name: S) -> ServiceHandle { ServiceHandle { name: name.into() } }

This allows calling with String, &str or several other types that can automatically be converted into a String. Making writing testing code with &'static str simple, while dynamically generated String objects still a first class citizen.


Categorieën: Mozilla-nl planet

Jean-Marc Valin: Opus 1.2 is out!

Mozilla planet - di, 20/06/2017 - 21:16
Opus gets another major upgrade with the release of version 1.2. This release brings quality improvements to both speech and music, while remaining fully compatible with RFC 6716. There are also optimizations, new options, as well as many bug fixes. This Opus 1.2 demo describes a few of the upgrades that users and implementers will care about the most. You can download the code from the Opus website.
Categorieën: Mozilla-nl planet

Air Mozilla: Rain of Rust -3rd online meeting

Mozilla planet - di, 20/06/2017 - 19:00

Rain of Rust -3rd online meeting This event belongs to a series of online Rust events that we run in the month of June, 2017

Categorieën: Mozilla-nl planet

Air Mozilla: Rain of Rust -3rd online meeting

Mozilla planet - di, 20/06/2017 - 19:00

Rain of Rust -3rd online meeting This event belongs to a series of online Rust events that we run in the month of June, 2017

Categorieën: Mozilla-nl planet

David Bryant: I think I know what you mean, Kev, and I understand, but if you ask me it is your day too.

Mozilla planet - di, 20/06/2017 - 17:04

I think I know what you mean, Kev, and I understand, but if you ask me it is your day too. Happy Father’s Day, from one father to another.

Categorieën: Mozilla-nl planet

Air Mozilla: Martes Mozilleros, 20 Jun 2017

Mozilla planet - di, 20/06/2017 - 17:00

Martes Mozilleros Reunión bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

Categorieën: Mozilla-nl planet

Air Mozilla: Martes Mozilleros, 20 Jun 2017

Mozilla planet - di, 20/06/2017 - 17:00

Martes Mozilleros Reunión bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

Categorieën: Mozilla-nl planet

The Mozilla Blog: Firefox Focus New to Android, blocks annoying ads and protects your privacy

Mozilla planet - di, 20/06/2017 - 15:06

Last year, we introduced Firefox Focus, a new browser for the iPhone and iPad, designed to be fast, simple and always private. A lot has happened since November; and more than ever before, we’re seeing consumers play an active role in trying to protect their personal data and save valuable megabytes on their data plans.

While we knew that Focus provided a useful service for those times when you want to keep your web browsing to yourself, we were floored by your response  – it’s the highest rated browser from a trusted brand for the iPhone and iPad, earning a 4.6 average rating on the App Store.

Today, I’m thrilled to announce that we’re launching our Firefox Focus mobile app for Android.

Like the iPhone and iPad version, the Android app is free of tabs and other visual clutter, and erasing your sessions is as easy as a simple tap.  Firefox Focus allows you to browse the web without being followed by tracking ads which are notoriously known for slowing down your mobile experience.  Why do we block these ad trackers? Because they not only track your behavior without your knowledge, they also slow down the web on your mobile device.

Check out this video to learn more:

 

New Features for Android

For the Android release of Firefox Focus, we added the following features:

  • Ad tracker counter – For the curious, there’s a counter to list the number of ads that are blocked per site while using the app.
  • Disable tracker blocker – For sites that are not loading correctly, you can disable the tracker blocker to quickly take care of it and get back to where you’ve left off.
  • Notification reminder – When Focus is running in the background, we’ll remind you through a notification and you can easily tap to erase your browsing history.

For Android users we also made Focus a great default browser experience. Since we support both custom tabs and the ability to disable the ad blocking as needed, it works great with apps like Facebook when you just want to read an article without being tracked. We built Focus to empower you on the mobile web, and we will continue to introduce new features that make our products even better. Thanks for using Firefox Focus for a faster and more private mobile browsing experience.

 

Firefox Focus Settings View

Firefox Focus Settings View

You can download Firefox Focus on Google Play and in the App Store.

The post Firefox Focus New to Android, blocks annoying ads and protects your privacy appeared first on The Mozilla Blog.

Categorieën: Mozilla-nl planet

QMO: Firefox 55 Beta 4 Testday, June 23rd

Mozilla planet - di, 20/06/2017 - 14:58

Hello Mozillians,

We are happy to let you know that Friday, June 23rd, we are organizing Firefox 55 Beta 4 Testday. We’ll be focusing our testing on the following new features: Screenshots and Simplify Page.

Check out the detailed instructions via this etherpad.

No previous testing experience is required, so feel free to join us on #qa IRC channel where our moderators will offer you guidance and answer your questions.

Join us and help us make Firefox better!

See you on Friday!

Categorieën: Mozilla-nl planet

Firefox Nightly: Resolved: Fixed – A short story about a community-reported bug

Mozilla planet - di, 20/06/2017 - 11:14

Firefox Nightly users, thanks to the telemetry an crash reports they send to Mozilla, are an amazing help to Firefox developers. The aggregated data sent by our community is extremely useful and allows spotting performance or stability regressions at the earliest stages of development. It probably can’t be emphasized enough how just using Nightly is a great way to get involved in Mozilla.

That said, we have in our Nightly community people that also actively hunt an report bugs and regressions and provide us detailed feedback, usually via the opening of a bug report in Bugzilla. These people are our core community, our first line of defense against regressions, they allow us to ship faster and better software and many of them have been involved in Mozilla and Firefox for a long time.

Mozilla Suite nightly start screen

Have you filed a bug?” is something you often hear open source developers say to people that report them some anomaly or regression, and for the majority of our users, this sounds like a complicated process. Just explaining the bug they experience in terms that make the bug report actionable by the developer that will fix it is a skill in itself. And this is where our core community of power-users on Nightly shines, they have this skill.

But what if the reporter has these skills but is not comfortable communicating in English because it is not her native language? Yes, language can also be a barrier to giving feedback…

A few days ago, a mozillian from our Spanish community (web developer in an IT company in Spain), sent me an email about a regression he was experiencing at work with nightly in the last days. This is the story I want to tell because it illustrates how powerful community work in open source can be and how lucky Mozilla is to have a dedicated global community.

Fernando, or StripTM as we know him in the Mozilla Hispano community, sent me an email about a major performance regression with forms on a page they have on their intranet, clicking in a form field would freeze the browser for seconds, and he wanted to know if I had heard about it. I didn’t so I asked him if there was a way I could see this page.

Intranets are tricky, the content there is by definition not public. But StripTM being a Web developer, he emailed me a reduced anonymized version of the page so as that I could test locally if I could see the bug and yes, I was experiencing it as well.

Since StripTM is not always comfortable writing bug reports in English, I did it for him and filed bug 1372843 a week ago and attached his test case. I fired up mozregression and found out that the bug was caused by the recent activation of Form Autofill in Nightly (see our article Preview Form Autofill in Firefox Nightly). In a nutshell, the performance problem was caused by the fact that this intranet page had 170(!) forms and our heuristics were cycling through all of the input fields in the page instead of only  the ones for the form we had clicked in.

All in all, it took a total of 3 days to discover the performance problem, file a bug and get a patch for it in mozilla-central. This is what happens when you can put passionate and skilled volunteers in contact with our equally passionate and skilled staff!

So thank you Fernando for using Nightly all these years and yes, the publishing date of this post is also a way for us to thank you for your involvement in Mozilla and wish you a happy birthday!

Categorieën: Mozilla-nl planet

Daniel Stenberg: c-ares 1.13.0

Mozilla planet - di, 20/06/2017 - 08:15

The c-ares project may not be very fancy or make a lot of noise, but it steadily moves forward and boasts an amazing 95% code coverage in the automated tests.

Today we release c-ares 1.13.0.

This time there’s basically three notable things to take home from this, apart from the 20-something bug-fixes.

CVE-2017-1000381

Due to an oversight there was an API function that we didn’t fuzz and yes, it was found out to have a security flaw. If you ask a server for a NAPTR DNS field and that response comes back crafted carefully, it could cause c-ares to access memory out of bounds.

All details for CVE-2017-1000381 on the c-ares site.

(Side-note: this is the first CVE I’ve received with a 7(!)-digit number to the right of the year.)

cmake

Now c-ares can optionally be built using cmake, in addition to the existing autotools setup.

Virtual socket IO

If you have a special setup or custom needs, c-ares now allows you to fully replace all the socket IO functions with your own custom set with ares_set_socket_functions.

Categorieën: Mozilla-nl planet

This Week In Rust: This Week in Rust 187

Mozilla planet - di, 20/06/2017 - 06:00

Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community News & Blog Posts Crate of the Week

This week's crate is include_dir, a crate that lets you include entire directory contents in your binary – like include_str!, but on steroids. Thanks to Michael Bryan for the suggestion!

Submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from Rust Core

122 pull requests were merged in the last week.

New Contributors
  • Marco Castelluccio
  • Thomas Lively
  • Wonwoo Choi
Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now. This week's FCPs are:

New RFCs Style RFCs

Style RFCs are part of the process for deciding on style guidelines for the Rust community and defaults for Rustfmt. The process is similar to the RFC process, but we try to reach rough consensus on issues (including a final comment period) before progressing to PRs. Just like the RFC process, all users are welcome to comment and submit RFCs. If you want to help decide what Rust code should look like, come get involved!

We're making good progress and the style is coming together. If you want to see the style in practice, check out our example or use the Integer32 Playground and select 'Proposed RFC' from the 'Format' menu. Be aware that implementation is work in progress.

Issues in final comment period:

Good first issues:

We're happy to mentor these, please reach out to us in #rust-style if you'd like to get involved

Upcoming Events

If you are running a Rust event please add it to the calendar to get it mentioned here. Email the Rust Community Team for access.

Rust Jobs

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week impl Clone for T { fn clone(&self) -> T { unsafe { std::ptr::read(self) } } }

@horse_rust on twitter.

Thanks to llogiq for the suggestion.

Submit your quotes for next week!

This Week in Rust is edited by: nasa42, llogiq, and brson.

Categorieën: Mozilla-nl planet

Mozilla Marketing Engineering & Ops Blog: MozMEAO SRE Status Report - June 20, 2017

Mozilla planet - di, 20/06/2017 - 02:00

Here’s what happened on the MozMEAO SRE team from June 13th - June 20th.

Current work Static site hosting
  • The irlpodcast site now has a staging environment also hosted in S3 with CloudFront. Additionally, Jenkins has been updated to deploy to staging and production via git push.

  • We’re going to move viewsourceconf.org from Kubernetes to S3 and CloudFront hosting. Production and staging environments have been provisioned, but we’ll need to update Jenkins to push changes to these new environments.

Basket move to Kubernetes Kubernetes (general)

Our DataDog, New Relic and MIG DaemonSets have been configured to use Kubernetes tolerations to schedule pods on master nodes. This allows us to capture metrics from K8s master nodes in additional to worker nodes.

Frankfurt Kubernetes cluster provisioning

Work continues to enable our apps in the new Frankfurt Kubernetes cluster. In addition, we’re working on automating our app installs as must as possible.

MDN
  • ElasticSearch will be upgraded to 2.4 in SCL3 production, June 21 11 AM PST

  • We may reconsider self-hosting ElasticSearch.

Links
Categorieën: Mozilla-nl planet

Daniel Stenberg: “OPTIONS *” with curl

Mozilla planet - ma, 19/06/2017 - 23:30

curl is arguably a “Swiss army knife” of HTTP fiddling. It is one of the available tools in the toolbox with a large set of available switches and options to allow us to tweak and modify our HTTP requests to really test, debug and torture our HTTP servers and services.

That’s the way we like it.

In curl 7.55.0 it will take yet another step into this territory when we finally introduce a way for users to send “OPTION *” and similar requests to servers. It has been requested occasionally by users over the years but now the waiting is over. (brought by this commit)

“OPTIONS *” is special and peculiar just because it is one of the few specified requests you can do to a HTTP server where the path part doesn’t start with a slash. Thus you cannot really end up with this based on a URL and as you know curl is pretty much all about URLs.

The OPTIONS method was introduced in HTTP 1.1 already back in RFC 2068, published in January 1997 (even before curl was born) and with curl you’ve always been able to send an OPTIONS request with the -X option, you just were never able to send that single asterisk instead of a path.

In curl 7.55.0 and later versions, you can remove the initial slash from the path part that ends up in the request by using –strip-path-slash. So to send an OPTION * to example.com for http and https URLs, you could do it like:

$ curl --strip-path-slash -X OPTIONS http://example.com/* $ curl --strip-path-slash -X OPTIONS https://example.com/*

In classical curl-style this also opens up the opportunity for you to issue completely illegal or otherwise nonsensical paths to your server to see what it does on them, to send totally weird options to OPTIONS and similar games:

$ curl --strip-path-slash -X OPTIONS http://example.com/*never* $ curl --strip-path-slash http://example.com/allpasswords

Enjoy!

Categorieën: Mozilla-nl planet

Chris McDonald: Message Broker: Goals and (De)Motivations

Mozilla planet - ma, 19/06/2017 - 22:57

Recently, I read a twitter rant that described message brokers as poor combination load balancer, database, and service discovery tools. It hit me hard since I’d just spent a week diving into writing my own message broker. While I had my dislikes of brokers, I think they are handy tools. The tweeter stated that many of these things should be built into the services. The goal of which to keep the heavy work out of the center of the system. Message brokers doing the opposite when used as a central bus.

Having this description of the problem space is turning out to be nice. It gives me some different framing for the various parts of the message broker I’ll be building and the underlying needs. It also pointed out a heavy flaw that message brokers as a central bus can cause trouble in some systems. While that twitter rant dismayed me at first, I now feel even more energized in building this tool.

This framing of load balancer, database, and service discovery reminds me to go read up on that tech as well. Sourcing papers for those problems while looking into queuing related things. I can acknowledge and make sure these subproblems get solved well enough for my intended scale. That will be a key part of my design going forward, keeping my decisions favoring small to medium scale. I’ve seen message brokers work well in those scenarios and want to make an even better one of those.

This doesn’t mean one couldn’t use the broker in a larger scale operation. But, I’m architecting it to encourage deliberate clustering beyond medium scale. Clustering acknowledges the fact that there are usually groups of services that are able to meet a work request without speaking outside of their group except for one or two edges. What I hope to discover as part of the development process is how to encourage this. Whether documenting and creating examples will be enough, or if I’ll need more core features.

I think keeping the message broker light weight will be instrumental in encouraging clustering. If the message broker is heavy, folks wouldn’t want to run too many instances. If it requires a lot of tuning to be useful, folks will want to only tune it once as a central bus. Side note: as I typed this I realized this is why Redis is so good.

Among the lofty design and architecture goals I want to mention my motivations and put the goals in perspective. This project’s main goal is to be a learning project. I want to better understand the internals of message buses. Most green field backend projects will be utilizing a message bus and smaller services. Understanding the internals of the message bus and keeping them in mind will let me design better services.

I also want to build a complex, performance focused, realistic piece of software in Rust. I find the language fun to work with and writing my own thread orchestration that is safe is delightful. As I build up the basics in the broker and client, I’m learning a lot of practical Rust skills. Like many others writing and coding in Rust in their free time, I’m hoping this will help encourage more jobs writing Rust. If I’m lucky enough, I’ll get to secure one of those jobs.


Categorieën: Mozilla-nl planet

Air Mozilla: Community Participation Guidelines Revision Brownbag (NALA)

Mozilla planet - ma, 19/06/2017 - 22:00

Community Participation Guidelines Revision Brownbag (NALA) A revised version of Mozilla's Community Participation Guidelines was released in May 2017. Please join Larissa Shapiro (Head of D&I) and Lizz Noonan (D&I Coordinator)...

Categorieën: Mozilla-nl planet

Air Mozilla: Community Participation Guidelines Revision Brownbag (NALA)

Mozilla planet - ma, 19/06/2017 - 22:00

Community Participation Guidelines Revision Brownbag (NALA) A revised version of Mozilla's Community Participation Guidelines was released in May 2017. Please join Larissa Shapiro (Head of D&I) and Lizz Noonan (D&I Coordinator)...

Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 19 Jun 2017

Mozilla planet - ma, 19/06/2017 - 20:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 19 Jun 2017

Mozilla planet - ma, 19/06/2017 - 20:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Pagina's