Mozilla Nederland LogoDe Nederlandse

Ανεβαίνουν κατακόρυφα οι επιδόσεις του Mozilla στα games (vid) -

Nieuws verzameld via Google - di, 14/03/2017 - 03:11

Ανεβαίνουν κατακόρυφα οι επιδόσεις του Mozilla στα games (vid)
Σε αυτό το θέμα έρχεται να δώσει τέλος o Mozilla καθώς πλέον είναι διαθέσιμη μια έκδοσή του για PC, Mac και Linux που προσφέρει “μοντέρνα” γραφικά στον browser σας χρησιμοποιώντας μια νέα τεχνολογία με την ονομασία WebAssembly. Μπορείτε να πάρετε ...

Categorieën: Mozilla-nl planet

Mitchell Baker: The “Worldview” of Mozilla

Mozilla planet - ma, 13/03/2017 - 21:59

There are a set of topics that are important to Mozilla and to what we stand for in the world — healthy communities, global communities, multiculturalism, diversity, tolerance, inclusion, empathy, collaboration, technology for shared good and social benefit.  I spoke about them at the Mozilla All Hands in December, if you want to (re)listen to the talk you can find it here.  The sections where I talk about these things are at the beginning, and also starting at about the 14:30 minute mark.

These topics are a key aspect of Mozilla’s worldview.  However, we have not set them out officially as part of who we are, what we stand for and how we describe ourselves publicly.   I’m feeling a deep need to do so.

My goal is to develop a small set of principles about these aspects of Mozilla’s worldview. We have clear principles that Mozilla stands for topics such as security and free and open source software (principles 4 and 7 of the Manifesto).  Similarly clear principles about topic such as global communities and multiculturalism will serve us well as we go forward.  They will also give us guidance as to the scope and public voice of Mozilla, spanning official communications from Mozilla, to the unofficial ways each of us describes Mozilla.

Currently, I’m working on a first draft of the principles.  We are working quickly, as quickly as we can have rich discussions and community-wide participation. If you would like to be involved and can potentially spend some hours reviewing and providing input please sign up here. Jascha and Jane are supporting me in managing this important project.  
I’ll provide updates as we go forward.  

Categorieën: Mozilla-nl planet

Chris Cooper: Shameless self (release) promotion: Firefox 53.0b1 from TaskCluster

Mozilla planet - ma, 13/03/2017 - 20:17

You may recall two short months ago when we moved Linux and Android nightlies from buildbot to TaskCluster. Due to the train model, this put us (release engineering) on a clock: either we’d be ready to release a beta version of Firefox 53 for Linux and Android using release promotion in TaskCluster, or we’d need to hold back our work for at least the next cycle, causing uplift headaches galore.

I’m happy to report that we were able to successfully release Firefox 53.0b1 for Linux and Android from TaskCluster last week. This is impressive for 3 reasons:

  1. Mac and Windows builds were still promoted from buildbot, so we were able to seamlessly integrate the artifacts of two different continuous integration (CI) platforms.
  2. The process whereby nightly builds are generated has always been different from how we generate release builds. Firefox 53.0b1 represents the first time a beta build was generated using the same taskgraph we use for a nightly, thereby reducing the delta between CI builds and release builds. More work to be done here, for sure.
  3. Nobody noticed. With all the changes under the hood, this may be the most impressive achievement of all.

A round of thanks to Aki, Johan, Kim, and Mihai who worked hard to get the pieces in place for Android, and a special shout-out to Rail who handled the Linux beta while also dealing with the uplift requirements for ESR52. Of course, thanks to everyone else who has helped with the migration thus far. All of that foundational work is starting to pay off.

Much more to do, but I look forward to updating you about Mac and Windows progress soon.

Categorieën: Mozilla-nl planet

The “Worldview” of Mozilla

Mitchell Baker - ma, 13/03/2017 - 19:28

There are a set of topics that are important to Mozilla and to what we stand for in the world — healthy communities, global communities, multiculturalism, diversity, tolerance, inclusion, empathy, collaboration, technology for shared good and social benefit.  I spoke about them at the Mozilla All Hands in December, if you want to (re)listen to the talk you can find it here.  The sections where I talk about these things are at the beginning, and also starting at about the 14:30 minute mark.

These topics are a key aspect of Mozilla’s worldview.  However, we have not set them out officially as part of who we are, what we stand for and how we describe ourselves publicly.   I’m feeling a deep need to do so.

My goal is to develop a small set of principles about these aspects of Mozilla’s worldview. We have clear principles that Mozilla stands for topics such as security and free and open source software (principles 4 and 7 of the Manifesto).  Similarly clear principles about topic such as global communities and multiculturalism will serve us well as we go forward.  They will also give us guidance as to the scope and public voice of Mozilla, spanning official communications from Mozilla, to the unofficial ways each of us describes Mozilla.

Currently, I’m working on a first draft of the principles.  We are working quickly, as quickly as we can have rich discussions and community-wide participation. If you would like to be involved and can potentially spend some hours reviewing and providing input please sign up here. Jascha and Jane are supporting me in managing this important project.  
I’ll provide updates as we go forward.  

Categorieën: Mozilla-nl planet

Advertiser tracking concerns second only to hacking for web users ... - The Drum

Nieuws verzameld via Google - ma, 13/03/2017 - 19:19

The Drum

Advertiser tracking concerns second only to hacking for web users ...
The Drum
Hacking is a lead concern for web users, but it is closely followed by advertiser tracking, an issue the marketing industry may have to address if it wants to ...
Mozilla: people have no idea how to protect their privacy and ...BetaNews

alle 2 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 13 Mar 2017

Mozilla planet - ma, 13/03/2017 - 19:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Doar un român din zece mai folosește Mozilla Firefox! - Capital

Nieuws verzameld via Google - ma, 13/03/2017 - 18:07


Doar un român din zece mai folosește Mozilla Firefox!
Dacă ani de zile toată lumea folosea Internet Explorer(IE), în ultimii ani Chrome, lansat de Google, domină autoritar piaţa browserelor. Pe plan local, 64,9% dintre români folosesc Chrome, în timp ce 11,55% Firefox, 7,34% Safari, 3.29% Samsung Internet ...

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: WebExtensions in Firefox 54

Mozilla planet - ma, 13/03/2017 - 18:00

Firefox 54 landed in Developer Edition this week, so we have another update on WebExtensions for you. In addition to new APIs to help more developers port over to WebExtensions, we also announced a new Office Hours Support schedule where developers can get more personalized help with the transition.

New APIs

A new API for creating sidebars was implemented. This allows you to place a local HTML file inside the sidebar. The API is similar to the one in Opera. If you specify the sidebar_action manifest key, Firefox will create a sidebar:

To allow keyboard commands to be sent to the sidebar, a new _execute_sidebar_action was added to the commands API which allows you trigger the showing of the sidebar.

The ability to override the about:newtab with pages inside your extension was added to the chrome_url_overrides field in the manifest. Check out the example that uses the topSites API to show the top sites you visit .

The privacy API gives you the ability to flip certain Firefox preferences related to privacy. Although the preferences in Chrome and Firefox aren’t a direct mapping, we’ve mapped the Firefox preferences that makes sense to the APIs. Currently implemented are: networkPredictionEnabled, webRTCIPHandlingPolicy and hyperlinkAuditingEnabled.

The protocol_handler API lets you easily map protocols to actions in your extension. For example: we use irccloud at Mozilla, so we can map ircs:// links to irccloud by adding this into an extension:

"protocol_handlers": [ { "protocol": "ircs", "name": "IRC Mozilla Extension", "uriTemplate": "!/%s" } ]

When a user clicks on an IRC link, it shows the application selector with the IRC Mozilla Extension visible:

This release also marks the landing of the first sets of devtools APIs. Quite a few APIs landed including: inspectedWindow.reload(), inspectedWindow.eval(), inspectedWindow.tabId, network.onNavigated, and panels.create().

Here’s an example of the Redux DevTools extension running on Firefox:

Backwards incompatible changes

The webRequest API will now require that you’ve requested the appropriate hosts’ permissions before allowing you to perform webRequest operations on a URL. This will be a backwards-incompatible change for any extension which used webRequest but did not request the host permission.

Deletes in storage.sync are now encrypted. This would be a breaking change for any extensions using storage.sync on Developer Edition.

API Changes

Some key events were completed in some popular APIs:

  • webRequest.onBeforeRequest is initiated before a server side redirect is about occur and webRequest.onAuthRequired is fired when an authentication failure occurs. These allow you to catch authentication requests from servers, such as proxy authentication.
  • webNavigation.onCreatedNavigationTarget event has been completed. This is fired when a new window or tab is created to be navigated to.
  • runtime.onMessageExternal event has been implemented. This is fired when a message is sent from another extension.

Other notable bugs include:


Notably in Firefox 54, basic tabs API support was landed for Android. The API support focuses on the parts of the API that make sense for Android, so some tab methods and events are deliberately not implemented.

This is an important API in its own right, but other key APIs did not have good support without this. By landing this, Android WebExtensions got much better webNavigation and webRequest support. This gives us a clear path to getting ad blockers, the most common extension type on Android.


A big thank you to our contributors Rob Wu, Tomislav Jovanovic and Tushar Saini who helped out with this release.

The post WebExtensions in Firefox 54 appeared first on Mozilla Add-ons Blog.

Categorieën: Mozilla-nl planet

Mozilla: people have no idea how to protect their privacy and security online - BetaNews

Nieuws verzameld via Google - ma, 13/03/2017 - 17:33

Mozilla: people have no idea how to protect their privacy and security online
Privacy and security are major concerns when it comes to life online, but a survey by Mozilla reveals that a worrying number of people do not know how to stay in control of them. The company also found that a third of people feel they have no control ...

Categorieën: Mozilla-nl planet

Waterfox, un fork de Firefox pour les déçus de Mozilla - MacGeneration

Nieuws verzameld via Google - ma, 13/03/2017 - 17:06


Waterfox, un fork de Firefox pour les déçus de Mozilla
Waterfox est en quelque sorte la version de Firefox pensée pour les déçus de Mozilla. Ce fork n'intègre ni Pocket ni le module DRM d'Adobe (qui va bientôt être remplacé dans Firefox par un module similaire de Google) et ne recueille aucune donnée ...

Categorieën: Mozilla-nl planet

Tim Guan-tin Chien: Ben’s Story of Firefox OS

Mozilla planet - ma, 13/03/2017 - 15:50

Like my good old colleague Ben Francis, I too have a lot to say about Firefox OS.

It’s been little over a year since my team and I moved away from Firefox OS and the ill-fated Connected Devices group. Over the course of last year, each time I think about the Firefox OS experience, I arrived at different conclusions and complicated, sometimes emotional, belives. I didn’t feel I am ready to take a snapshot of these thoughts and publish it permanently, so I didn’t write about it here. Frankly, I don’t even think I am ready now.

In his post, Ben has pointed out many of the turning points of the project. I agree with many of his portrayals, most importantly, lost of direction between being a product (which must ship fast and deliver whatever partners/consumers wanted and used to) and a research project (which involves engineering endeavors that answer questions asked in the original announcement). I, however, have not figured out what can be done instead (which Ben proposed in his post nicely).

Lastly, a final point: to you, this might as well be another story in the volatile tech industry, but to me, I felt the cost of people enormously whenever a change was announced during the “slow death” of Firefox OS.

People moves on and recovers, including me (which fortunately wasn’t nearly being hit the hardest). I can only extend my best wishes to those who had fought the good fight with.

Categorieën: Mozilla-nl planet

Gregory Szorc: from __past__ import bytes_literals

Mozilla planet - ma, 13/03/2017 - 10:55

Last year, I simultaneously committed one of the ugliest and impressive hacks of my programming career. I haven't had time to write about it. Until now.

In summary, the hack is a source-transforming module loader for Python. It can be used by Python 3 to import a Python 2 source file while translating certain primitives to their Python 3 equivalents. It is kind of like 2to3 except it executes at run-time during import. The main goal of the hack was to facilitate porting Mercurial to Python 3 while deferring having to make the most invasive - and therefore most annoying - elements of the port in the canonical source code representation.

For the technically curious, it works as follows.

The hg Python executable registers a custom meta path finder instance. This entity is invoked during import statements to try to find the module being imported. It tells a later phase of the import mechanism how to load that module from wherever it is (usually a .py or .pyc file on disk) to a Python module object. The custom finder only responds to requests for modules known to be managed by the Mercurial project. For these modules, it tells the next stage of the import mechanism to invoke a custom SourceLoader instance. Here's where the real magic is: when the custom loader is invoked, it tokenizes the Python source code using the tokenize module, iterates over the token stream, finds specific patterns, and rewrites them to something more appropriate. It then untokenizes back to Python source code then falls back to the built-in loader which does the heavy lifting of compiling the source to Python code objects. So, we have Python 2 source files on disk that magically get transformed to be Python compatible when they are loaded by Python 3. Oh, and there is no performance penalty for the token transformation on subsequence loads because the transformed bytecode is cached in the .pyc file (using a custom header so we know it was transformed and can be invalidated when the transformation logic changes).

At the time I wrote it, the token stream manipulation converted most string literals ('') to bytes literals (b''). In other words, it restored the Python 2 behavior of string literals being bytes and not unicode. We jokingly call it from __past__ import bytes_literals (a play on Python 2's from __future__ import unicode_literals special syntax which changes string literals from Python 2's str/bytes type to unicode to match Python 3's behavior).

Since I implemented the first version, others have implemented:

As one can expect, when I tweeted a link to this commit, many Python developers (including a few CPython core developers) expressed a mix of intrigue and horror. But mostly horror.

I fully concede that what I did here is a gross hack. And, it is the intention of the Mercurial project to undo this hack and perform a proper port once Python 3 support in Mercurial is more mature. But, I want to lay out my defense on why I did this and why the Mercurial project is tolerant of this ugly hack.

Individuals within the Mercurial project have wanted to port to Python 3 for years. Until recently, it hasn't been a project priority because a port was too much work for too little end-user gain. And, on the technical front, a port was just not practical until Python 3.5. (Two main blockers were no u'' literals - restored in Python 3.3 - and no % formatting for b'' literals - restored in 3.5. And as I understand it, senior members of the Mercurial project had to lobby Python maintainers pretty hard to get features like % formatting of b'' literals restored to Python 3.)

Anyway, after a number of failed attempts to initiate the Python 3 port over the years, the Mercurial project started making some positive steps towards Python 3 compatibility, such as switching to absolute imports and addressing syntax issues that allowed modules to be parsed into an AST and even compiled and loadable. These may seem like small steps, but for a larger project, it was a lot of work.

The porting effort hit a large wall when it came time to actually make the AST-valid Python code run on Python 3. Specifically, we had a strings problem.

When you write software that exchanges data between machines - sometimes machines running different operating systems or having different encodings - and there is an expectation that things work the same and data roundtrips accordingly, trying to force text encodings is essentially impossible and inevitably breaks something or someone. It is much easier for Mercurial to operate bytes first and only take text encoding into consideration when absolutely necessary (such as when emitting bytes to the terminal in the wanted encoding or when emitting JSON). That's not to say Mercurial ignores the existence of encodings. Far from it: Mercurial does attempt to normalize some data to Unicode. But it often does so with a special Python type that internally stores the raw byte sequence of the source so that a consumer can choose to operate at the bytes or Unicode level.

Anyway, this means that practically every string variable in Mercurial is a bytes type (or something that acts like a bytes type). And since string literals in Python 3 are the str type (which represents Unicode), that would mean having to prefix almost every '' string literal in Mercurial with b'' in order to placate Python 3. Having to update every occurrence of simple primitives that could be statically transformed automatically felt like busy work. We wanted to spend time on the meaningful parts of the Python 3 port so we could find interesting problems and challenges, not toil with mechanical conversions that add little to no short-term value while simultaneously increasing cognitive dissonance and quite possibly increasing the odds of introducing a bug in Python 2. In other words, why should humans do the work that machines can do for us? Thus, the source-transforming module importer was born.

While I concede what Mercurial did is a giant hack, I maintain it was the correct thing to do. It has allowed the Python 3 port to move forward without being blocked on the more tedious and invasive transformations that could introduce subtle bugs (including performance regressions) in Python 2. Perfect is the enemy of good. People time is valuable. The source-transforming module importer allowed us to unblock an important project without sinking a lot of people time into it. I'd make that trade-off again.

While I won't encourage others to take this approach to porting to Python 3, if you want to, Mercurial's source is available under a GPL license and the custom module importer could be adapted to any project with minimal modifications. If someone does extract it as reusable code, please leave a comment and I'll update the post to link to it.

Categorieën: Mozilla-nl planet

Chrome en Firefox bestempelen miljoenen HTTP-websites als onveilig -

Nieuws verzameld via Google - ma, 13/03/2017 - 09:01

Chrome en Firefox bestempelen miljoenen HTTP-websites als onveilig
Miljoenen websites wereldwijd worden getroffen door een beslissing van Google en Mozilla om in de webbrowsers Chrome en Firefox om http-websites als onveilig te bestempelen. Het gaat onder andere om websites van populaire media, apothekers en ...

Categorieën: Mozilla-nl planet

The Servo Blog: These Weeks In Servo 94

Mozilla planet - ma, 13/03/2017 - 01:30

In the last two weeks, we landed 185 PRs in the Servo organization’s repositories.

Planning and Status

Our overall roadmap is available online, including the overall plans for 2017 and Q1. Please check it out and provide feedback!

This week’s status updates are here.

Notable Additions
  • samgiles and rabisg added the Origin header to fetch requests.
  • hiikezoe made CSS animations be processed by Stylo.
  • Manishearth supported SVG presentation attributes in Stylo.
  • ferjm allowed redirects to occur after a CORS preflight fetch request.
  • sendilkumarn corrected the behaviour of loading user scripts in iframes.
  • pcwalton improved the performance of layout queries and requestAnimationFrame.
  • emilio removed unnecessary heap allocation for some CSS parsers.
  • mephisto41 implemented gradient border support in WebRender.
  • jdm avoided some panics triggered by image elements initiating multiple requests.
  • MortimerGoro improved the Android integration and lifecycle hooks.
  • nox removed the last uses of serde_codegen.
  • ferjm avoided a deadlock triggered by the Document.elementsFromPoint API.
  • fitzgen improved the rust-bindgen support for complex template parameter usages.
  • gw implemented page zoom support in WebRender.
  • dpyro added support for the nosniff algorithm in the Fetch implementation.
  • KiChjang implemented the :lang pseudoclass.
New Contributors

Interested in helping build a web browser? Take a look at our curated list of issues that are good for new contributors!

Categorieën: Mozilla-nl planet

Mike Hommey: When the memory allocator works against you

Mozilla planet - zo, 12/03/2017 - 02:47

Cloning mozilla-central with git-cinnabar requires a lot of memory. Actually too much memory to fit in a 32-bits address space.

I hadn’t optimized for memory use in the first place. For instance, git-cinnabar keeps sha-1s in memory as hex values (40 bytes) rather than raw values (20 bytes). When I wrote the initial prototype, it didn’t matter that much, and while close(ish) to the tipping point, it didn’t require more than 2GB of memory at the time.

Time passed, and mozilla-central grew. I suspect the recent addition of several thousands of commits and files has made things worse.

In order to come up with a plan to make things better (short or longer term), I needed data. So I added some basic memory resource tracking, and collected data while cloning mozilla-central.

I must admit, I was not ready for what I witnessed. Follow me for a tale of frustrations (plural).

I was expecting things to have gotten worse on the master branch (which I used for the data collection) because I am in the middle of some refactoring and did many changes that I was suspecting might have affected memory usage. I wasn’t, however, expecting to see the clone command using 10GB(!) memory at peak usage across all processes.

(Note, those memory sizes are RSS, minus “shared”)

It also was taking an unexpected long time, but then, I hadn’t cloned a large repository like mozilla-central from scratch in a while, so I wasn’t sure if it was just related to its recent growth in size or otherwise. So I collected data on 0.4.0 as well.

Less time spent, less memory usage… ok. There’s definitely something wrong on master. But wait a minute, that slope from ~2GB to ~4GB on the git-remote-hg process doesn’t actually make any kind of sense. I mean, I’d understand it if it were starting and finishing with the “Import manifest” phase, but it starts in the middle of it, and ends long before it finishes. WTH?

First things first, since RSS can be a variety of things, I checked /proc/$pid/smaps and confirmed that most of it was, indeed, the heap.

That’s the point where you reach for Google, type something like “python memory profile” and find various tools. One from the results that I remembered having used in the past is guppy’s heapy.

Armed with pdb, I broke execution in the middle of the slope, and tried to get memory stats with heapy. SIGSEGV. Ouch.

Let’s try something else. I reached out to objgraph and pympler. SIGSEGV. Ouch again.

Tried working around the crashes for a while (too long while, retrospectively, hindsight is 20/20), and was somehow successful at avoiding them by peaking at a smaller set of objects. But whatever I did, despite being attached to a process that had 2.6GB RSS, I wasn’t able to find more than 1.3GB of data. This wasn’t adding up.

It surely didn’t help that getting to that point took close to an hour each time. Retrospectively, I wish I had investigated using something like Checkpoint/Restore in Userspace.

Anyways, after a while, I decided that I really wanted to try to see the whole picture, not smaller peaks here and there that might be missing something. So I resolved myself to look at the SIGSEGV I was getting when using pympler, collecting a core dump when it happened.

Guess what? The Debian python-dbg package does not contain the debug symbols for the python package. The core dump was useless.

Since I was expecting I’d have to fix something in python, I just downloaded its source and built it. Ran the command again, waited, and finally got a backtrace. First Google hit for the crashing function? The exact (unfixed) crash reported on the python bug tracker. No patch.

Crashing code is doing:

((f)->f_builtins != (f)->f_tstate->interp->builtins)

And (f)->f_tstate is NULL. Classic NULL deref.

Added a guard (assessing it wouldn’t break anything). Ran the command again. Waited. Again. SIGSEGV.

Facedesk. Another crash on the same line. Did I really use the patched python? Yes. But this time (f)->f_tstate->interp is NULL. Sigh.

Same player, shoot again.

Finally, no crash… but still stuck on only 1.3GB accounted for. Ok, I know not all python memory profiling tools are entirely reliable, let’s try heapy again. SIGSEGV. Sigh. No debug info on the heapy module, where the crash happens. Sigh. Rebuild the module with debug info, try again. The backtrace looks like heapy is recursing a lot. Look at %rsp, compare with the address space from /proc/$pid/maps. Confirmed. A stack overflow. Let’s do ugly things and increase the stack size in brutal ways.

Woohoo! Now heapy tells me there’s even less memory used than the 1.3GB I found so far. Like, half less. Yeah, right.

I’m not clear on how I got there, but that’s when I found gdb-heap, a tool from Red Hat’s David Malcolm, and the associated talk “Dude, where’s my RAM?” A deep dive into how Python uses memory (slides).

With a gdb attached, I would finally be able to rip python’s guts out and find where all the memory went. Or so I thought. The gdb-heap tool only found about 600MB. About as much as heapy did, for that matter, but it could be coincidental. Oh. Kay.

I don’t remember exactly what went through my mind then, but, since I was attached to a running process with gdb, I typed the following on the gdb prompt:

gdb> call malloc_stats()

And that’s when the truth was finally unvealed: the memory allocator was just acting up the whole time. The ouput was something like:

Arena 0: system bytes = some number above (but close to) 2GB in use bytes = some number above (but close to) 600MB

Yes, the glibc allocator was just telling it had allocated 600MB of memory, but was holding onto 2GB. I must have found a really bad allocation pattern that causes massive fragmentation.

One thing that David Malcolm’s talk taught me, though, is that python uses its own allocator for small sizes, so the glibc allocator doesn’t know about them. And, roughly, adding the difference between RSS and what glibc said it was holding to to the use bytes it reported somehow matches the 1.3GB I had found so far.

So it was time to see how those things evolved in time, during the entire clone process. I grabbed some new data, tracking the evolution of “system bytes” and “in use bytes”.

There are two things of note on this data:

  • There is a relatively large gap between what the glibc allocator says it has gotten from the system, and the RSS (minus “shared”) size, that I’m expecting corresponds to the small allocations that python handles itself.
  • Actual memory use is going down during the “Import manifests” phase, contrary to what the evolution of RSS suggests.

In fact, the latter is exactly how git-cinnabar is supposed to work: It reads changesets and manifests chunks, and holds onto them while importing files. Then it throws away those manifests and changesets chunks one by one while it imports them. There is, however, some extra bookkeeping that requires some additional memory, but it’s expected to be less memory consuming than keeping all the changesets and manifests chunks in memory.

At this point, I thought a possible explanation is that since both python and glibc are mmap()ing their own arenas, they might be intertwined in a way that makes things not go well with the allocation pattern happening during the “Import manifest” phase (which, in fact, allocates and frees increasingly large buffers for each manifest, as manifests grow in size in the mozilla-central history).

To put the theory at work, I patched the python interpreter again, making it use malloc() instead of mmap() for its arenas.

“Aha!” I thought. That definitely looks much better. Less gap between what glibc says it requested from the system and the RSS size. And, more importantly, no runaway increase of memory usage in the middle of nowhere.

I was preparing myself to write a post about how mixing allocators could have unintended consequences. As a comparison point, I went ahead and ran another test, with the python allocator entirely disabled, this time.

Heh. It turns out glibc was acting up all alone. So much for my (plausible) theory. (I still think mixing allocators can have unintended consequences.)

(Note, however, that the reason why the python allocator exists is valid: without it, the overall clone took almost 10 more minutes)

And since I had been getting all this data with 0.4.0, I gathered new data without the python allocator with the master branch.

This paints a rather different picture than the original data on that branch, with much less memory use regression than one would think. In fact, there isn’t much difference, except for the spike at the end, which got worse, and some of the noise during the “Import manifests” phase that got bigger, implying larger amounts of temporary memory used. The latter may contribute to the allocation patterns that throw glibc’s memory allocator off.

It turns out tracking memory usage in python 2.7 is rather painful, and not all the tools paint a complete picture of it. I hear python 3.x is somewhat better in that regard, and I hope it’s true, but at the moment, I’m stuck with 2.7. The most reliable tool I’ve used here, it turns out, is pympler. Or rebuilding the python interpreter without its allocator, and asking the system allocator what is allocated.

With all this data, I now have some defined problems to tackle, some easy (the spike at the end of the clone), and some less easy (working around glibc allocator’s behavior). I have a few hunches as to what kind of allocations are causing the runaway increase of RSS. Coincidentally, I’m half-way through a refactor of the code dealing with manifests, and it should help dealing with the issue.

But that will be the subject of a subsequent post.

Categorieën: Mozilla-nl planet

Eric Rahm: Are they slim yet, round 2

Mozilla planet - vr, 10/03/2017 - 19:16

A year later let’s see how Firefox fares on Windows, Linux, and OSX with multiple content processes enabled.


Graph comparing memory usage, chrome is still quite high

We can see that Firefox with four content processes fares better than Chrome on all platforms which is reassuring; Chrome is still about 2X worse on Windows and Linux. Our current plan is to only move up to four content processes, so this is great news.

Two content processes is still better than IE, with four we’re a bit worse. This is pretty impressive given last year we were in the same position with one content process.

Surprisingly on Mac Firefox is better than Safari with two content processes, compared with last year where we used 2X the memory with just one process, now we’re on par with four content processes.

I included Firefox with eight content processes to keep us honest. As you can see we actually do pretty well, but I don’t think it’s realistic to ship with that many nor do we currently plan to. We already have or are adding additional processes such as the plugin process for Flash and the GPU process. These need to be taken into consideration when choosing how many content processes to enable and pushing to eight doesn’t give us much breathing room. Making sure we have measurements now is important; it’s good to know where we can improve.

Overall I feel solid about these numbers, especially considering where we were just a year ago. This bodes well for the e10s-multi project.

Test setup

This is the same setup as last year. I load the first 30 pages of the tp5 page set (a snapshot of Alexa top 100 websites from a few years ago), each in its own tab, with 10 seconds in between loads and 60 seconds of settle time at the end.

Note: There was a minor change to the setup to give each page a unique domain. At least Safari and Chrome are roughly doing process per domain, so just using different ports on localhost was not enough. A simple solution was to modify my /etc/hosts file to add localhost-<1-30> aliases.


Measuring multiprocess browser memory usage is tricky. I’ve settled with a somewhat simple formula of:

total_memory = sum_uss(content processes) + sum_rss(parent processes);

Where a parent process is defined as anything that is not a content process (I’ll explain in a moment). Historically there was just one parent process that manages all other processes, this is still somewhat the case but each browser still has other executables they may run in addition to content processes. A content process has a slightly different definition per browser, but is generally “where the pages are loaded” — this is an oversimplification, but it’s good enough for now.

My definitions:

Browser Content Definition Example “parent” Firefox firefox processes launched with the -contentproc command line. firefox without the -contentproc command line, plugin-process which is used for Flash, etc. Chrome chrome processes launched with the --type command line. chrome without out the --type command line, nacl_helper, etc. Safari WebContent processes. Safari, SafariServices, SafariHistory, Webkit.Networking, etc. IE iexplore.exe process launched with the /prefetch command line. iexplore without the /prefetch command line. Edge MicrosoftEdgeCP.exe processes. MicrosoftEdge.exe, etc.

For Firefox this is a reasonable and fair measurement, for other browsers we might be under counting memory by a bit. For example Edge has a parent executable, MicrosoftEdge.exe, and a different content executable, MicrosoftEdgeCP.exe, arguably we should measure the RSS of one the MicrosoftEdgeCP.exe processes, and USS for the rest, so we’re probably under counting. On the other hand we might end up over counting if the parent and content processes are sharing dynamic libraries. In future measurements I may tweak how we sum the memory, but for now I’d rather possibly under count rather then worry about being unfair to other browsers.

Raw numbers OS Browser Total Memory Ubuntu 16.04 LTS Chrome 54 (see note) 1,478 MB Ubuntu 16.04 LTS Firefox 55 – 2 CP 765 MB Ubuntu 16.04 LTS Firefox 55 – 4 CP 817 MB Ubuntu 16.04 LTS Firefox 55 – 8 CP 990 MB macOS 10.12.3 Chrome 59 1,365 MB macOS 10.12.3 Firefox 55 – 2 CP 1,113 MB macOS 10.12.3 Firefox 55 – 4 CP 1,215 MB macOS 10.12.3 Firefox 55 – 8 CP 1,399 MB macOS 10.12.3 Safari 10.2 (see note) 1,203 MB Windows 10 Chrome 59 1,382 MB Windows 10 Edge (see note) N/A Windows 10 Firefox 55 – 2 CP 587 MB Windows 10 Firefox 55 – 4 CP 839 MB Windows 10 Firefox 55 – 8 CP 905 MB Windows 10 IE 11 660 MB Browser Version Notes
  • Chrome 54 — aka chrome-unstable — was used on Ubuntu 16.04 LTS as that’s the latest branded version available (rather than Chromium)
  • Firefox Nightly 55 – 2 CP is Firefox with 2 content processes and one parent process, the default configuration for Nightly.
  • Firefox Nightly 55 – 4 CP is Firefox with 4 content processes and one parent process, this is a longer term goal.
  • Firefox Nightly 55 – 8 CP is Firefox with 8 content processes and one parent process, this is aspirational, a good sanity check.
  • Safari Technology Preview 10.2 release 25 was used on macOS as that’s the latest branded version available (rather than Webkit nightly)
  • Edge was disqualified because it seemed to bypass the hosts file and wouldn’t load pages from unique domains. I can do measurements so I might revisit this, but it wouldn’t have been a fair comparison as-is.
Categorieën: Mozilla-nl planet

Gervase Markham: Firefox Secure Travel Addon

Mozilla planet - vr, 10/03/2017 - 18:15

In these troubled times, business travellers occasionally have to cross borders where the border guards have significant powers to seize your electronic devices, and even compel you to unlock them or provide passwords. You have the difficult choice between refusing, and perhaps not getting into the country, or complying, and having sensitive data put at risk.

It is possible to avoid storing confidential data on your device if it’s all in the cloud, but then your browser is logged into (or has stored passwords for) various important systems which have lots of sensitive data, so anyone who has access to your machine has access to that data. And simply deleting all these passwords and cookies is a) a pain, and b) hard to recover from.

What might be very cool is a Firefox Secure Travel addon where you press a “Travelling Now” button and it:

  • Disconnects you from Sync
  • Deletes all cookies for a defined list of domains
  • Deletes all stored passwords for the same defined list of domains

Then when you arrive, you can log back in to Sync and get your passwords back (assuming it doesn’t propagate the deletions!), and log back in to the services.

I guess the border authorities can always ask for your Sync password but there’s a good chance they might not think to do that. A super-paranoid version of the above would also:

  • Generate a random password
  • Submit it securely to a company-run web service
  • On receiving acknowledgement of receipt, change your Sync password to
    the random password

Then, on arrival, you just need to call your IT department (who would ID you e.g. by voice or in person) to get the random password from them, and you are up and running. In the mean time, your data is genuinely out of your reach. You can unlock your device and tell them any passwords you know, and they won’t get your data.

Worth doing?

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: Improvements to add-on review communications

Mozilla planet - vr, 10/03/2017 - 14:00

We recently made some improvements to our tools and processes to better support communication between add-on developers and reviewers.

Previously, when you submitted an add-on to (AMO) and a reviewer emailed you, your replies went to a mailing list (amo-editors AT mozilla DOT org) where a few reviewers (mostly admins) handled every response. This approach had some flaws—it put the burden on very few people to reply, who had to first get familiar with the add-on code and previous review actions. Further replies from either party went to the mailing list only, rather than being fed back into the review tools on AMO. These flaws slowed things down unnecessarily, and contributed to information clutter.

Now, add-on developers can choose to reply to a review by email—like they’re used to—or from the Manage Status & Versions page of the add-on in the developer hub. Replies are picked up by AMO and displayed in the review history for reviewers and developers. In addition, everyone  involved in the review of the particular version will be notified by email. Admin reviewers will make sure all inquiries are followed up with.

This long-anticipated feature will not only make follow-ups for reviews more efficient for both developers and reviewers, it also makes upcoming reviews easier by having all information in the same place.

The mailing list (amo-editors AT mozilla DOT org) will be discontinued shortly, so we ask all developers to use this system instead. For other questions not related to a particular review, please send a message to amo-admins AT mozilla DOT org.

The Add-on Review team would like to thank Andrew Williamson for implementing this new feature and our QA team for testing it!

The post Improvements to add-on review communications appeared first on Mozilla Add-ons Blog.

Categorieën: Mozilla-nl planet

Daniel Glazman: OS X TouchBar

Mozilla planet - vr, 10/03/2017 - 13:13

Currently adding OS X TouchBar support to XUL :-) So far so good. Should be trivial to add multiple touchbars to a given XUL window when it's done.

Categorieën: Mozilla-nl planet

Karl Dubost: [worklog] Edition 058 - Rain, sign of spring

Mozilla planet - vr, 10/03/2017 - 10:05
webcompat life webcompat issues dev
  • By introducing HTTP Caching to our HTML resources, I also introduced an issue. The bug report says that people can't login, logout. In fact they can. The issue is what the page looks like, it doesn't display that the user is logged-in. It's normal I initial put a max-age of one day in addition to the etag value. In browser speak, it means that if the browser has a cached copy it will not even try a conditional requests with If-None-Match. Because during the login process we are hitting the same resource. Users are receiving the previous cached copy and don't see that they have actually logged in. A force reload solves that. So instead of putting a max-age I can solve this with a no-cache. Unfortunately, browsers didn't respect the HTTP semantics of no-cache and decided to make it a no-store. Back to the departure case. must-revalidate, max-age=0 will solve it and force the conditional request.
  • Opened quick issue about our current twitter link based on the UX research currently done.
  • Discussions about issues with dependencies and how do we handle them?
  • Discussing about the Contact page friendliness
  • We currently have some outreachy participants. Time for review and help: review
  • Our issues title are… pretty poor for now.


Categorieën: Mozilla-nl planet