Braintrust query: Tom Critchlow wants to run River5 on Glitch. Has anyone had any success with this?#
A new kind of software I want to do. Starting with Node.js on the server and JavaScript in the browser as the foundation, ship sample apps or code examples that solve common problems with a neat simple API. We did this inside Frontier for many years, so there's plenty of prior art. The first one is a simple feed reading API. It takes the URL of a feed and calls back with either an error, or a JavaScript object containing the content of the feed, in a standard form, flattening the differences between the formats. Your code doesn't care. It also handles charset encoding. It's as simple as it can be. Now there's no excuse for not adding a feed reading capability to your Node app. #Without a strong tech press we can't have open formats and protocols, because the big tech companies will just usurp them, monetize them, squeeze all the juice out of them, and leave what remains to rot. We don't have journalism today that watches and reports on the power grabs of the tech industry. That's why it's so easy to expose corruption. It's lying around in the open for all to see. But if you look deeper you'll see how it all connects. No reporter has yet had the will or the requisite technology background to do it.#
I got a confusing email this morning from Flickr, saying I'd have to migrate my site to SmugMug by May 25, or download and delete my site. Then I read their FAQ that said I would only be agreeing to new terms and privacy policy. In 2014 I wrote a post saying someday Flickr will die. Today is not that day, but it certainly raises the question again. I don't know much about SmugMug. Not the greatest name ever. Might want to just call it Flickr, a better-known name that has been on shaky ground for a long time. Saying it's part of SmugMug now probably doesn't do much to add confidence. Just sayin. #
I finished all sixteen episodes of Bablylon Berlin. It was quite a ride. And at the end I'm not sure exactly what happened. I needed this summary to convince me that I did/do. Would I recommend it? It's a roller coaster. Dark. Some scenes are hard to watch. The song is addictive. You fall in love with the characters. Would I watch it again? Not sure. But the ending isn't exactly Singin' in the Rain. 💃🕺#
Don't you know the best things in life are free? It's true!#
It's incredibly sad that New York magazine thinks the internet was the creation of venture capitalists and execs. Next they're going to ask us to believe that love was created by VCs. The heavens and stars above. Life itself.#
New York magazine quotes Wikimedia exec: "There is no public internet, and we are the closest thing to it," which is of course complete nonsense. You know all those citations at the end of every Wikipedia article? Where do you think those come from? Wikimedia owes the internet an apology. And New York mag, I used to respect it, has been running a series with the theme that the internet wants to apologize. That's like saying the Grand Canyon wants to apologize. This is what journalism thinks it can get away with. No different than the garbage Fox News sells. #
It's great that the Dems sued Trump, Wikileaks, Russians if for no other reason that they will have to give depositions. Firing Mueller is looking like less of a solution to anything. Also pardon power does not extend to civil penalties. Heh. Good lawyering.#For such a famous place, it's weird that Silicon Valley doesn't have a landmark. Paris has the Eiffel Tower. NYC has the Empire State Building, London has Big Ben. I remember arriving in Silicon Valley in 1979 and driving around looking for it and finding nothing but suburbs. I guess that's why Buck's was so popular. It certainly wasn't the food. 💥#
On a discussion board, I posted, in response to a fellow techie talking about interfaces that work for grandmas. "Here we are, a bunch of middle-aged men, with no clue wtf is going on. On the other hand things are so broken this way, there's no chance of it getting fixed. I don't like the NYC subways either, but they're what we got. Technology is like evolution, it favors what works, and never fixes its mistakes."#
- James Comey is a lawyer and bureaucrat.#
- He doesn't have that much to say.#
- He was spectacularly wrong about something really important, and doesn't know it. #
- And he is no Michael Wolff, a muck-raker and rabble-rouser, by profession.#
- If you want an idea of why no one told you what Facebook was up to, look no further than the press. It was their job to tell you, after the tech companies. #
- Here's the lead paragraph of a news story written by John Markoff in the NY Times on this day in 2015.#
- "Silicon Valley has a richly deserved reputation as the world’s engine of technology innovation, with a track record that includes developing integrated circuits, microprocessors, personal computers and smartphones. This is a culture of confidence and bravado. Ask a bunch of tech leaders about their goals, and it’s a good bet that many of them will utter the words, 'To change the world.'"#
- That was and probably still is the way the press views the tech industry. Until they get over it, don't expect much reality from them re tech.#
- A new TV show format. Tours of neighborhoods in various parts of the US. Show people in different parts how we live, and vice versa.#
- Walk through a typical supermarket and show what you can buy and what the prices are.#
- The nearest airport.#
- An average commute.#
- See it as a person living there would see it.#
- Confront perceptions with reality.#
- Reality TV that is real reality.#
As of April 11, you had no excuse for not knowing that the privacy scandal was much bigger than Cambridge Analytica and Facebook. #
This song is the point in Babylon Berlin when I knew I was hooked. It was about three episodes in, and i can see now that many of the main characters are in this scene. I didn't know that the first time around. #
Sometimes by chance Twitter pairs two messages that totally belong together. Death is very real, and unambiguous. People who care about language should leave it to do it's work when it's what's really happening. Speaking as someone with recent experience with actual death. And I doubly hate it when that word is applied to something I care about.#
Anyway not only is blogging not xxx, but look at all the new features here on Scripting News. Many of these things have never been done in blogs before, or for that matter on news org sites. You only believe it's xxx if you ignore everything outside your little piece of the blogging world (which for many is limited to their own blog). Just as we need to branch out and share innnovations in the land of RSS, we must do that for blogging too, so that when people say it's xxx, we can show them how it's anything but. You want blogging to thrive? It can, if you're willing to look, and tell the story.#Maybe one of the problems is that the main blogging platforms aren't moving. Or if they are moving, only in increments, and not in features that readers can appreciate. What a blog is isn't changing much. And that's not good, or necessary. #
I found a feedBase problem, an interaction with the new checkboxes, de-duping and dereferencing feed URLs. It would manifest this way: Click a checkbox for a feed, reload the page, the feed is unchecked. But only for a few feeds. For most feeds it worked as it should (that's why I didn't catch the problem the first time around).#- The common denominator -- the feeds were one of the de-duped feeds on the hotlist. The solution is to be careful with the de-duping map, to always map to the one that's preferred by the server, because we deref the URL before subscribing. We weren't doing that for a few of the de-duped feeds. The problem may come up with future mappings and I want to be sure we don't have to repeat the debugging process. #
- Another thing -- when dereferencing a URL, if the only difference is the protocol, don't use the deref, stick with what you have.#
- I'm beginning to realize that we need feeds to have a guid, to take all the guesswork out of this. It's a real mess! Once you try to maintain a database of feeds, something I've not actually done myself before, you buy into trying to come up with a canonical ID for a feed. The URL works pretty well, until you realize that there are several different URLs for each feed. #
- Also realizing we should have popped the protocol off the URL before using it as a key so http://xxx would be the same feed as https://xxx.#
Wouldn't it be a great to have a blog that covered developments across all feed readers, so we can follow what's new? If you make a reader, would you support this? Not sure who would be good to write it, but we'd need someone who loves RSS, and isn't in business.#I was able to replace iconv with iconv-lite in River5, so if you have been having trouble installing, it makes sense to try again. #
Example code: Using iconv-lite to do character conversion of HTTP requests. Here is the original request for help#
When I was growing up in NYC, before the EPA, they'd burn garbage in incinerators, in the morning, as I walked to school. It was choking. My mom sent me out clean, by the time I got to school there'd be burnt garbage in my hair and on my clothes. #
- Little-known fact: I designed and developed a programming language.#
- My goal was to create an environment I would work in for the rest of my career. I just realized it's exactly 30 years later, and I'm still using it.#
- 30 fucking years. I think I earned the right to say it that way. 🚀#
- Now that I also work in JavaScript, it amazes me how easy the simple things are in Frontier, compared to JS, esp when you have to tack on a database. You really have to work at seeing what's going on. In Frontier, you just click around expanding things. You can even look at the runtime stack that way.#
- We hacked and optimized and reworked things. In comparison, JS was developed in a more chaotic way. Too many ways to do the same thing imho. In Frontier there tended to be one, which is my preference. However there are missing features. Maybe at some point I'll try to rebuild Frontier in JavaScript. I think I'm now good enough to do it. The original was written in straight C.#
- Where would I start? db.c of course. 💥#
- PS: Most people don't know about Frontier. But you probably do know about things that were developed in Frontier. Like the first blogs, podcasts, RSS feeds, readers and content creation tools, XML-RPC and lots of other good shit. People would ask me how I got so much done. "Great tools." That's Frontier.#
Imagine a Twitter where the limit wasn't expressed in the number of characters, but in the number of vertical pixels.#
A friend who uses Feedly told me about his collection of NYT feeds. He sent a pointer, where Feedly offered me a chance to use his list. But only on Feedly. Ugh. Ideally, they would let me have the list in OPML form. I suspect they do, but if so, it's not obvious esp to someone who would want the list in OPML, i.e. someone who is not a frequent Feedly user, such as yours truly. This is the problem with commercial vendors who build on open formats. They don't reciprocate. They consume the openness. This happened before. Even better if they offered a dynamic link to the user's collection, so I could add the link to the OPML to my reader and if his collection changed I'd be updated. Remember the big idea of the web, people return to places that send them away. Feedly is doing the opposite, trying to suck people in and hold them. This is the tech industry philosophy, and it will imho be its downfall.#"Hannity has no ethics" is not news. #
I would pay a fee to subscribe to a group of news pubs. I think this is necessary, but this should be created and managed independent of the tech industry. News already looks to tech to be its sugar daddy which leads to fawning coverage, and a huge conflict, at a time when tech is more in the news than ever, and deserving of scrutiny. Tech controlling news flow, especially Apple, which has little respect for criticism, free speech, spells the end of any semblance of independence of news. Also I'm hearing more that paywalls are seriously stifling the flow of news, at a time when we need better flow. Apple would not be good for that either. #
I'm thinking about getting a new iPad, and said so on Twitter. I got a bunch of responses, including this blog post from Matt Ballantine, who loves the iPad because of its compatibility with Apple's pencil. Based on his report, I decided to get the new iPad and the pencil. I used to be a diagram person, as part of pitching ideas to other people, I'd develop what I called a chalk talk. A very good way to communicate, highly personal and persuasive. #- Ariel Anbar posted a caveat about the pencil on Facebook. #
- My main problem with the Pencil is that it doesn’t hold a charge long when unused - maybe a couple of days - and recharging it requires an additional step beyond my nightly recharge of the iPad. So unless I get into a daily routine with it, I find it is usually dead in the moment of inspiration-driven need.#
- On the plus side, it recharges very quickly, but 5 min is an eternity when you suddenly have the need. Add to this the occasional need to mess around to get it to reconnect, and it is one of those really cool and tantalizing and useful but not-quite-up-to-its-lofty-potential pieces of tech.#
- As for how to carry it, there are many iPad cases that have solutions. Some are quite inexpensive.#
- Hmmm. That's too bad. I wondered why Apple didn't promote the product more, maybe this is why. Even so, I think I'll give it a try. #
18-minute podcast about the Denver newspapers and Berkeleyside. The Denver news orgs are doing something unusual, crossing the wall between publishing and editorial. And Berkeleyside, a local news org who just did a public offering of stock, and eliminated the wall between publishing and editorial. Have a listen and think if perhaps this isn't a better way forward for news than paywalls and hedge-fund ownership.#
I wish the Democrats were self-aware enough to choose the strongest candidate to run against Trump or whoever is the Republican nominee in 2020. People still think in terms of a perfect candidate. Dems don't have any. But that's not a problem because the Repubs don't have any either. Imho what we need is someone who appeals to the massive core of American voters, without betraying us to the super-rich. Never mind who appeals to you -- you're going to vote rationally. Think about the massive number of voters who don't. I don't think they're fascists or KKKs, I think they are emotional and want to feel good. Who can give a rousing even angry speech that gets people to feel good about themselves without tearing other people down (except for Trump of course). That's the one. Imho#Sometimes things have to fall apart before they can start to make sense again. When they write the history of the web, if they get the story right, a big event will be the San Francisco newspaper strike in 1994. A lot of ideas came together there, across a wide divide. It showed us how news and tech are inextricably entwined.#
The best part of the Comey interview last night was when he asked what the f*ck were Russians doing in the Oval Office, without any Americans there other than the president (who is obviously a Russian tool).#
- There have been a couple of reports of trouble installing the latest River5, both having to do with the iconv package, which is a relatively new addition to River5. Here's the latest report.#
- I don't know what the problem is, I had no problems installing it on my Mac or on a Linux server. #
- When I have trouble with NPM, this is what I do:#
- Delete the node_modules folder and do an npm install. #
- If that doesn't work, clear NPM's cache, delete the node_modules folder and do an npm install. #
- What's unusual about iconv is that it's written in C, and as part of the npm install process it needs to compile it to machine code. #
- I am anything but an expert in NPM problems, that's why I'm raising a flag here on Scripting News. #
- Update: I think Anton got to the bottom of it. Some systems have it set up so that you can't run downloaded stuff without modifying permissions. #
As you know from reading this blog, I am a big fan of efforts to make the web long-lived. And that's why I was interested in this story about how the NYT is creating new archives of old stories so they appear on the web exactly as they did when the stories ran. #- For example, here's an archive of the NYT home page for 9/11/2001. Interestingly, I took a screen shot of that page earlier in the day, as the story was unfolding. There was no question history was happening that day.#
- It's good, but who is going to do this for historic weblogs? I've kept my blog around, and various experiments I've done over the years. I have generally tried to use technology that I believed was going to stick around, so I've never built on Flash, for example. I've used static HTML files as much as possible. But even so, there are quite a few gaps in my archive, esp where I have let domains lapse. #
- And of course a huge bonfire of breakage is coming as Google tries to turn off HTTP. This is something users of the web, news orgs, libraries, historians, researchers, should join me in condemning. Changing to a new protocol is fine if you want to do it, but trying to force people to? That's a company that needs to be told to stay in its lane. More on this on the Google and HTTP faq.#
Open source developers need as much support as the users of their products. With that in mind, I want to congratulate the developers of jsDelivr. I just adapted my project to use it as a CDN. The process was hard work, but made much easier by the docs and design of jsDelivr. That part was a breeze. They tell you just what you need to know when you need to know it with no extra fluff. This is really rare. Good work! 💥#BTW, development on feedBase is on pause, but I'm going to swing back around to it. I find it's good that once the initial problems are shaken out of a new product, it's good to step back and let it settle down before the next major construction project. (Update: as soon as I wrote this, I found two glaring problems. I'm going to fix them now, not going to wait for the next round of work.)#
I wrote an overview of Cambridge Analytica and Facebook, the piece I feel the press hasn't written. In case you haven't seen it. 🚀#
- I want a future-of-news conference where we plan new open systems for news publishing and reading, without sponsorship of big tech companies such as Facebook or Google. #
- There would be a session at the conference entitled How To Get Facebook and Google to Give Us Money, and it would be off-topic at every other session. That way the sessions wouldn't all be repetitive expressions of powerlessness and we could get some work done.#
- People who were at BloggerCon will recognize this as the How To Make Money With Your Blog session at that conference. We swept up all the powerlessness into one session, and made it off-topic at every other one. It worked.#
- Artists when they get together, all they want to talk about is how they need to make money. I've never seen a discussion among creative people that didn't immediately devolve to this. In F-O-N, big tech companies encourage this. They want all the attention focused on them.#
- The new releases of River5 and davereader are out.#
- It felt like it was going to be a small change, but it wasn't. #
- That said, there are still references to fargo.io in the code. That'll take more time to shake out because this code is shared with my other projects. But the main files for displaying rivers are now in jsdelivr.#
- Please update your River5 installation, watch for problems .#
- If you want to see all the code that's being accessed through the CDN, it's in the River5 repository on GitHub.#
Made a lot of progress today on using jsdelivr as a CDN for River5. Expect to have something to release tomorrow. #
- I posed a query on Twitter this morning. "Uber is to taxis as AirBnB is to hotels as x is for news. What is x?"#
- Jim Parsons said the answer is RSS. #
- To which I said: "Sorry that's not what I was getting at. RSS is a channel for news. The others are replacements for taxis and hotels. Not analogous."#
- Then I thought about it and changed my mind.#
Uber and AirBnB are channels, not the cars or homes that are shared. So RSS is perfectly analogous. The question I was really asking is this: Where is the analog in news for Uber drivers and AirBnB hosts. I once thought and still do that blogging was the seed for that movement. It faltered, perhaps because roadblocks were put in its way in the form of Google Reader and the pretense among news people that they were bloggers, and perhaps the relatively low value users place on news. We know what a taxi ride is worth, and a hotel room, but we have a less precise understanding of the value of news.#- I still think that blogging is the answer. I know news people find it abhorrent, but I think to the extent they do, they are actually anti-news people, like the Dutch boy with his finger in the dike. #
My personal privacy policy re Facebook -- everything I post is 100% public, as if I posted it to my blog. Same with Twitter.#
- Journalists say it's time for Facebook to endow journalism. I think that's a really bad idea.#
- Journalism has to compete with tech, not be supported by tech. The best of both. It's way too early to surrender. That's what asking for a handout is. #
- The best product in journalism and tech has yet to be created. It won't be if journalism is married to Facebook. How can Facebook's competitors emerge in this environment? They can't.#
- I want journalism to listen much better. For example, the story of Facebook and Cambridge Analytica could have been reported eight years ago. Any number of independent technologists could have helped journalists develop this story. But they don't answer the calls. What incentive will journalism have to change if the bills are being paid by Facebook? They're already too impressed with huge tech companies.#
- I want to compete with Facebook. All people with ideas in news and software should be able to. Senator Lindsey Graham asked Zuck the right question on Tuesday. Who are your competitors? The answer is they have none, though Zuck wouldn't say that. That is the big problem. Solve it and all the others go away. It requires Facebook to do a much better job of linking out from the main user experience. Meaningful change could happen immediately. Over a period of a few years, Facebook could open up to peering with competitive social networks. This would enable all kinds of new technology and news flows to develop. #
- I want to build a beautiful online world just for news, not to tack it on to a discussion board. Start from zero. Given the current wealth of resources on the net, what would you create for news? I know what I want to make, and it would not be Facebook. It would start at a different place.#
- It's too early to lock it down, either tech or journalism. Still too much change that needs to happen. Honestly I think it's the change that the incumbents in journalism are trying to avoid. The change could have started over 20 years ago. It's time to give up on the old way and create the new way. The ideas of Facebook definitely apply. But not Facebook itself. #
- PS: For background see yesterday's piece on where we're at with Facebook.#
- Here is my summary of what I've been writing on my blog for the last few weeks while the Cambridge Analytica/Facebook controversy was brewing.#
- There are two stories, advertising and world domination#
- The first story: targeted advertising. Facebook has an ad system, a web app like Facebook itself, where people enter credit card information, keywords and demographics to determine who sees their ad. They don't get any information about the users this way.#
- The second story: Facebook's API. This is how Cambridge Analytica got the data.#
- In 2010 Facebook announced with huge fanfare and press coverage that they were opening up Facebook's data to developers. They called it The Social Graph. They would accumulate every fact in the world, and turn that into a platform for developers.#
- They weren't doing this out of the goodness of their heart, though there is some of that (seriously, not kidding). This is how tech companies achieve dominance. They capture developers and potential competitors by being the environment everyone makes software for. We become part of their ecosystem, they control us. I'm sure Bill Gates and Steve Jobs schooled him on how to do this. It's how power is expressed in tech.#
- Was it a breach?#
- Equifax was a bonafide breach. The information captured was what you need to impersonate people in financial transactions. 148 million identities.#
- In 2008, the US government had a massive data breach of veterans data, 70 million records. Again the kind of data that can be used to impersonate people to steal money. #
- We're not sure all that has leaked through the Facebook API. In the last few days people have been saying that direct messages, which feel very private, were shared through the API. If so, yes this was a breach. However everything else was pretty well disclosed.#
- The reason it's a shock is the press failed to report on it in 2010 and after. Facebook does a thorough job of disclosing it to users. Zuckerberg was being dishonest in other ways in his testimony, but in this respect, he was being truthful. Not necssarily kind or fair, however, because they, like everyone in tech, are aware that users don't understand what they're giving up.#
- What's unique about Cambridge Analytica?#
- Not much. Every developer that has access to the Facebook API, and it's more or less open to the world, any of them could have done what CA did. The API been around for 8 years, and in that time companies formed solely to use the data that you can get through the API. Many of them. #
- An analogy. CA as a crisis would be like saying a leaky faucet in Manhattan is a crisis. The crisis could be all the leaky faucets everywhere. The intersection of CA with Facebook, while it sounds big is actually pretty small in comparison to the actual scope of the problem.#
- Were users hurt?#
- Because I am a developer who uses the same API as CA I knew what was going on. #
- Even so -- as a Facebook user -- I tried some of the data capture services that masquerade as tests and quizzes. But relative to Equifax, other credit companies, doctors I've used, banks, many of which have had real data breaches, Facebook really doesn't have much if any info that isn't already out there. #
- It's really hard to find much damage done to users here. Much less so than with Equifax or NARA, for example. #
- First Amendment#
- Facebook is a silo, and that's bad for freedom, but it is a place where a lot of speech happens, so there are First Amendment issues with regulating Facebook. #
- Should there be regulation?#
- Yes. On privacy disclosure, and prohibiting the most draconian uses of user data. It should not be possible for users to give those rights up in exchange for use of a social system like Facebook. The idea is similar to the law in California that says that most non-competes are not enforceable. The benefit you receive has to be somewhat equivalent to the data you give up. #
- What about Google, Apple, Amazon?#
- This is the really important stuff.#
- This affair should get users, government and the press to look at other tech companies who have business models based on getting users to disclose ever-more-intimate information. Here are some examples.#
- Google, through Android, knows every place you go. They use that data. Do they sell it? I don't know, but I'm pretty sure you can use it to target ads. Apple, through the iPhone also knows where you go.#
- Apps on Android or iPhones can be told where you go. Many of them are only useful if you let them have the info. Apps can also have all your pictures, contacts. Face recognition makes it possible to construct a social graph without any access to the Facebook API.#
- Google and Apple can listen to all your phone calls.#
- Google, through their Chrome browser, knows everywhere you go on the web, and everything you type into the browser. #
- Amazon Echo and Google Home are always listening. Imagine a leak based on conversations at home, phone calls, personal habits, arguments you have with your spouse, kids, any illegal activities that might be going on in your home. #
- If you have a Gmail account, Google reads your mail, and targets ads at you based on what you're writing about. They also read the email that people send to you, people who may not also be Gmail users. Some examples of how creepy this can be -- they seem to know what my investments are -- I assume they figured this out through email. Recently they told me when a friend's flight to NYC was arriving. I don't know how they made this connection. I assume it was through email.#
- Amazon, of course, knows everything you buy through Amazon. #
- Google knows everything you search for. #
- And on and on. We've reconstructed our whole society around companies having all the data about us that they want. It's kind of funny that we're all freaking out about Cambridge Analytica and Facebook. The problem is so much bigger. #
- Summary#
- It seems like a non-event to me. The press knew all about the API going back to 2010. That they didn't foresee the problem then is a result of the press accepting the hype of big tech companies on their terms, and not trying to find out what the implications of technology are from non-partisan experts. This was a story that could have and should have been written in 2010, warning users of a hidden cost to Facebook.#
- Today's scandal, the equivalent of the one in 2010, is that Google is attempting to turn the web into a corporate platform. Once they control the web as Facebook controls the Social Graph, we'll have another impossibly huge problem to deal with. Better to head this one off with regulation, now, when it can do some good. #
New feedBase feature. Checkboxes on feed lists. If it's checked you're subscribed, if not, you're not. Click click click. You're in charge. Easy and super-fast. 🐄#
Wait till the press figures out that Google knows everywhere you go. Not only on the web, but also in the world.#
A couple of technical/strategic questions came up yesterday that are still outstanding. First, I need to pick a CDN to use for River5 files that are used in user apps. Right now they're served on domains that I own that could go away. That would be a lot of breakage. Which one should I use? Second, I want to make some mods to the forever app we use to keep Node apps running. I need to figure out how to set it up so the new version installs the way forever currently installs. I don't want/need to do this differently, but it's outside my expertise. As always, help appreciated. #
Sarcasm has no place in serious discourse.#
The press is the real political power in the US.#
I now better-understand the concern of South Asians re Apu. They are getting bullied, reduced to stereotype. Apu is funny to us, but not to them.#- The analogy is not true. As a Jew, I've never had someone call me Krusty. If it happened I would have no idea what they were talking about. Not so with Apu and Indians.#
- We have trouble understanding the problem with Apu because we love him. But they don't love him. I'm not sure what to do about it, but the people who write The Simpsons said pretty much exactly the worst thing about it in the latest episode.#
If RSS was a startup and had a CEO that TechCrunch needed access to, or was backed by VCs, or even a big tech company, they would temper their judgment. Instead it's a punching bag, a technology piñata. But people love RSS. It's still here, ready to bring you the news. And honestly they probably wouldn't beat on it if it weren't threatening someone. ❤️#I have been playing with the source for the forever utility. I want to try adding a few features, see what happens. But I have a basic question.#
Dan MacTough answers the question I posed in my podcast, asking about how to treat URLs of feeds in a MySQL database.#
I've had Frontier running on my desktop for over a month. Using it all day every day for writing code and documenting stuff. I never noticed how reliable it is, even in 2018, many years after its last overhaul. A testimony to fine engineering by André Radke and Ted C Howard, two programmers who probably have never met. Just wanted to put a H/T out there and a big thank you. (In comparison, I have to restart my JavaScript apps every day or they are way too slow.)#
I have developed all kinds of JavaScript stuff and have never used promises. I like callbacks. They aren't too complicated for me. I know I'm probably the only one.#
Sometimes a company asks a great question on Facebook, and wtf, I answer it. Like this one. They asked us to name a concert that totally blew your mind. Here's what I wrote. "Beach Boys as the surprise guest act with the Dead at the Fillmore East in 1971. When they were setting up no one knew who they were, they had grown their hair, turned into hippies. Then they played a hit, Heroes and Villains, and the place came apart."#
I hate possessions. With my mom dying earlier this year, the house I grew up in will soon be out of the family. All this stuff, if we threw it out, would be gone forever. It was nice that she was keeping it all. Then I think of all the possessions I've taken from that house in the past. Where are they now? Not anywhere I can put my hands on them. I have shed all my possessions several times in my life. Yet I keep accumulating them. This year I got far more than I ever wanted. What will become of them? I really can't be trusted with things. ;-)#
Is journalism serving us? I doubt if this question comes up much in the hallowed halls of the New York Times. The answer to them is self-evident, of course news serves the people. #- What I, one of the people, see is something very different, an impenetrable wall that ideas and news can't enter, except for the ways of stature, access, and ad dollars, and occasionally something to do with the interests of the audience. #
- In 2016, just after the election, I wrote an angry piece about the press, which had just sold us out, giving us an idiot-despot as president. Now they're trying to rewrite history to say that Facebook did it. I know a lot of the facts, far more than what they report, and almost certainly more than the reporters do. I also watched the build-up to this campaign, where they openly ran editorials and op-eds saying that "public opinion" was about to turn against Facebook. That's insider-talk for "we're looking for a good opportunity to attack them." They found it in Cambridge Analytica, a non-scandal, non-story, but one that could be presented as one.#
- In journalism-land people will think what they tell us to think. #
- Dean Baquet, executive editor of the NYT, says this is a terrible time for journalism because local news is failing. There is a way out. Knock down the wall and let us in, while there still is a modicum of respect for the great old names of journalism. Let's grow journalism by an order of magnitude. There must be a million capable people who believe in free speech who are ready to write news, for free, for the glory of it, for the chance to set things straight, to tell the story as they see it, as it is actually happening. Of course, in order for that to happen, the people who currently write the news will have to take a demotion, from the originators of public opinion to teachers and editors, coaches and quality-assurers. Innovators. And historic adjusters of news to the realities of a world where anyone can write the news. #
- A world where anyone can write the news. That's both the weakness and the strength of news in this century. If there is no quality to the writing, we get Trump, a product of the current antiquated, vestigial, news system, the one the high priests of journalism wrongly tell us we must protect. We tried it their way, it didn't work. #
- Now we should try another approach. It will be different, it feel uncomfortable, but the job of journalism "is to comfort the afflicted and afflict the comfortable." Fate has a sense of humor because now the comfortable who are afflicted are the journalists themselves. #
New version of River5. After burning in the new feed reading package in feedBase, I made the same fix in the davereader package, which is the core of River5. If you were seeing garbled text in feeds in German, for example, this version should fix that. To update, quit the app, type npm update at the command line, then restart the app. #Brent Simmons rewrote his blogging software so to handle titleless posts. It's more significant because Brent is developing a feed reader that will do a nice job of supporting titleless posts. Eventually imho all readers will. #
It's amazing to me how slowly progress is made while people prove they don't have to listen to some other person. It is true. You don't have to listen to anyone (except a cop, or your mother). But you could listen, if you wanted to know what someone else thought? Would it kill you? 💥#
Facebook built a system for sharing information. When you grant access to your Facebook account to someone else, they get access to the info you shared. That's it. That's what's going on. If you gave access to Cambridge Analytica, they got the info you shared. When you give access to any other company, same deal. If you don't want them to have your info, don't grant them access. #
I love the way the Twitter devs joined up and respectfully documented a problem they were having with the Twitter API. No powerlessness. Just facts. And I love that Twitter read what they wrote, and said they'd work with them. Everyone involved deserves maximum credit.#
BTW lost in all the Trump hooplah, there was a successful teacher's march in West Virginia and Oklahoma, and one on the way in Arizona. The Open Source podcast covers it, as always, in a thoughtful and exciting way. #
The rule of links: "Linking is an art. It's a choice. You don't link from every word or even every noun, or from the subject of every sentence. But when a reader reasonably would want to know more about the subject, the Rule of Links says you should link to it." #
UX tip. I clicked on a wrong link in my bank website. I wanted to quickly get out. I clicked the Cancel button. A dialog appeared asking if I really want to cancel. Yes or no. This is dissonant. I do not want to click YES when I'm saying get me the fcuk out of here now.#
I have a new credit card. This morning I paid for a Starbuck's using Apple Pay. I expected it to fail because I hadn't updated the number. It worked. When I looked it already had the new number. Before I did.#
I generally don't like to look at how mailbox-type readers render this site. The best I've seen are ones that ignore titleless posts. At least they don't show the reader a mangled version of my writing. Then there's the ones that repeat the body of titleless posts in the title. So you see my writing twice, as if I had a stutter. It's not my stutter, the text only appears once in my feed. Readers that get on board with titleless posts are helping to open the door for new kinds of blogging. I've chosen to ignore their ignoring and just plow ahead. BTW, when I say "new way" I really mean "old way." My blog, the original one, esp for RSS support, had titleless posts years before RSS existed. And the format allows titleless posts. So those that don't support them can't really claim to support RSS. #BTW, here's an archive page from this blog in 1996. Look at how the format is evolving. Lots of title-less posts. It's a good format. A lot like Twitter, btw. 💥#
- Michael Uffer had a great idea. Link the orange feed icon next to every feed title to the user's favorite reader's page for that feed. #
- I added the feature, and was immediately able to get it to work with Feedly. Here's an example of a URL that we generate, to view the items it has for the Daring Fireball blog. #
- Then we got it to work with Inoreader. Same idea, here's a link to an Inoreader page for Kottke.#
- I haven't been able to find the equivalent URL for Old Reader, but if it's there, we'll find it. #
- Here's the thread.#
If you've been reading this blog for the last week or so, it must be very confusing. The result of my confusion. I finally had to understand encoding on the web. I wanted feedBase to work for feeds in all languages. The web went through a transition to make it work that way. At UserLand, Jake and Brent did the encoding stuff. I hoped that my new app, feedBase, would "just work." Didn't happen.#- There were two problems:#
- My connection to the feedparser package was not encoding-aware. This, btw, is also true in River5. Rather than do a solution just for feedBase, I did a general solution, coming up with the API that I wanted on Node, basically the same API that we have in Frontier. I will use that package in River5. Hopefully that will make the transition much easier than it otherwise would be. #
- My SQL database wasn't set up for Unicode. So I had to figure out the magic incantations to make it work that way. As with feed parsing, I had hoped it would just work. But I had some excellent help from the newly assembled MySQL mentorship group, again mostly consisting, of course, of JY. He's a real life-saver. #
- Anyway, I'm keeping my eye on it, but I'm hoping now to be able to move on. 💥#
For feedBase users, I'm interested in creating a connection with two or three readers as a proof of concept for a contemplated feature. So I'd just like to know what reader you use. That's all. 🌸#
Some credible j-schools, in conjunction with their CS departments, should hold boot camps for reporters when aspects of tech become hot news. Right now we need reporters to get up to speed on APIs.#
BTW, it seems Facebook's response to the heat it's getting is to kill the Instagram API and to limit the Facebook API. This is not a good thing for the web, or it may be a very good thing for the web. Hard to say. #
We're in a tight corner in the conversion of feedBase to Unicode support. But I'm sure there's an answer because anyone who has developed a feed reader with MySQL as the database has had to answer this question. I recorded a 17-minute podcast that explains the question in detail, and why this is a good time to slow down and get it right. The question: What should be the maximum length for the URL of a feed? But please listen before answering. Update: The problem appears to be solved. The trick was to not convert the URLs and shorten the username to 32 chars and convert that (because it can be unicode and it's part of the key for the subscriptions table). It will take some time to fix all the feeds, but it seems we have the problem solved. Knock wood. I can't believe it. I get to take a break now. Haha. Let's see. 🚀#Just learned maximum length for Twitter user names is 15 characters. That's good, believe it or not. 💥#
Good morning sports fans!#
More work today on feedBase. Yesterday I got the backup routine working, and released the code. Next, I'm going to get a copy of the database running on my desktop Mac. Then I'm going to try the conversion so it can handle UTF-8 characters. If it works, I'll do the same on the deployed version. Then this issue will be closed. #
Also, I fixed a bug yesterday that prevented feedBase from working in Safari, even though for some reason it worked for me in Safari. The bug was pretty egregious, and it should have prevented it from working anywhere. Mysterious. But fixed. #
All of a sudden I was surrounded by Hasidim on bikes with yellow flags. They were everywhere.#
- Re Facebook's social graph -- they chose to do something visionary. No one else was up to the challenge of such a huge database, and they pulled it off, at a huge ever-increasing scale. #
- In hindsight maybe it seems like a bad idea, but you'd have to know in advance that users would have no idea what was going on, and wouldn't care about privacy (until journalists told them they should). An open protocol was never going to get adopted, most individual devs like to reinvent, not build on others' work. #
- You won't often hear me defend Facebook, but please, some of the critics are seriously misinformed about how the tech works, and how we got here. Yes, it matters. #
I gravitate to writers who are in my experience always right. But when they are wrong, and eventually they are, it's spectacular how wrong they are. Not naming names, of course. 💥#
- Note: Of course the greater crime is shooting innocent workers at a tech company, there is no excuse for that. But it is worth noting why Nasim Najafi Aghdam attacked YouTube yesterday. #
- I read about the woman who shot up YouTube yesterday, and found her story sympathetic. Translated to the terminology of net architecture, she attacked a silo that dominated her online existence, both through YouTube and through Google's search engine. #
- I sympathize because Google lowers the rank of my blog because I use HTTP. As if that had anything to do with the authority of the information posted here (it doesn't). #
- And they complain that pages that were posted before there were mobile devices aren't mobile-friendly. How can I tell them how ridiculous that is? #
- And the ultimate humiliation, calling my sites NOT SECURE and their own sites SECURE. Nice going. Who appointed them arbiter of what is and isn't secure on the web? Where do I go to appeal this ruling? #
- I also don't like that they took down her videos after the shooting. In coming days, might they take down posts like this one beause it is critical of Google? Of course they will. #
- I am a non-violent person, they have nothing to fear from me on that count, but I think one lesson they could take from this experience is they should exercise as little control as possible over the content they serve, or get out of the content serving business. #
- #
It's good that people are discussing the Google Reader debacle. Important to not let Google repeat their bad behavior with whole web.#
Lots of responses to yesterday's query about backing up the feedBase MySQL database. I posed another question in the thread just now. How do you all feel about writing JSON files from Node.js code? Backing up is a prerequisite to all adventures that might risk the data in the database. #Braintrust question: Re MySQL and Unicode. I assumed the text would all be UTF-8, but apparently not so. I can use help/advice from the MySQL mentors. Thanks! :-)#
Demo of new feedBase feature: "Sometimes you have the URL of a feed, and don't want to go to the trouble of adding it to an OPML file so you can upload it. This command lets you just enter the URL to feedBase and it processes it as if it were part of an OPML file."#
So it follows from Trump's desperate tweeting re Bezos that the Washington Post must have him nailed but good.#
An experiment. I changed the text font on Scripting News today to Georgia 20. It had been Ubuntu 18. A little bigger and a serif font. I think this is easier to read. I always planned to take another look at this after things settled in on the new design. (Update: I undid the change. I prefer the more compact format.)#
Re the change I made yesterday to subscriber counts on user-feed-lists in feedBase, I changed it per this feature request. Now we sort by name but include the subscriber count. An example. I think this is the more rational approach. It's useful to see the subscriber counts. Note that the counts are not the combined counts, so you'll only see the counts for the specific feed URL you're subscribed to, not the combination of all the aliases.#
- Today on Twitter, former US Attorney Preet Bharara dreams: "What if Jeff Bezos bought Twitter with the change in his pocket and shut Trump's account?" #
- Twitter's market cap is $21 billion, and Bezos or some other very rich person could buy them. But it would be a problem if the new owner interfered with the content of Twitter. Because, as with a trade war, it would invite retaliation. And isn't it good that we can keep tabs on Trump through his public tweetings? Would you rather he only rant privately? I wouldn't. Remember the president has an incredibly powerful podium, with or without Twitter, he will be heard over us. Consider this piece by Jill McCabe, an emergency room pediatrician, married to former FBI deputy director Andrew McCabe, as she defends her family from attacks from the president. Consider the power Trump has with or without Twitter to ruin lives. #
- I responded, on Twitter: "It's just as easy to imagine the Koch Brothers buying Twitter and shutting down your account. This is a weakness that we all have with Twitter being so vulnerable yet so indispensable." #
- PS: I've long wanted Amazon to either buy or create a competitor to Twitter, so we could have Twitter-as-a-service. It would be very nice. #
Another feedBase change. When viewing a user's feed list, we sort the feeds alphabetically by title, and don't include the follower count. It makes sense on the hotlist, but not when looking at the feeds an individual user subscribes to.#
We now merge duplicate items on the feedBase hotlist. So if there are two ways to reference a feed, we add the two counts, and remove one of them. I did it by hand, by looking for duplicates, added code that does the merging. If you spot other duplicates please let me know. #
I need a MySQL mentor or two. While I was working on BingeWorthy and early versions of feedBase, I had two really good mentors, JY Stervinou and Scott Hanson. I don't want to lean just on them, so I'm putting out a call for a few people with MySQL experience so I can ask questions, and get answers based on experience. Saves a lot of time, and it will make feedBase more robust. With that in mind I have a basic question today, posted here. Thanks in advance! 💥#
Usually when I criticize a professional journalist they ignore it. Not very often they respond. I don't mind if they disagree, and push back, but I most definitely do mind if they question the idea of my criticizing their work at all. Last week I had such an encounter, privately. #- The criticism of a blogger, a user, someone who isn't trying to maintain access, or please advertisers, is imho more likely to locate a truth about your product, from their point of view, of course.. I never maintain that mine is the only valid point of view, just that it is mine. Take it for what it's worth.#
- I also have been wrong many times. Famously when desktop publishing was the rage in the mid-80s, I said no one wanted to do it. Hah. It turned out that everyone did, at least at the time. #
- That said, I've also been right. #
- I love it when people take the time to respond to what I write, or give me a good bug report on my software. As long as it's thoughtful and makes a point. I even appreciate the effort when they don't. But to outright invalidate the criticism without response? Only scoundrels do that. 💥#
- I've written here, a couple of times, why I like Axios. #
- Today I did a little video demo of Axios and Fast Company, to illustrate.#
- Axios gets to the point right away, in the first paragraph. And they name names, in a skimmable way. It's designed to be read on the web, quickly.#
- Other publications, such as Fast Company, have evolved in a different way. Their goal is to keep you reading, by never getting to the point. I almost always give up before figuring out what it was. Which has trained me not to click on FC links.#
- Here's the demo.#
Number of feeds in feedBase: 25452. #
Version 0.6.2 of the feedBase server. Uses new functionality of davefeedread, so we should handle feeds that use different charsets. Also dereference the feed URL before adding a subscription. This should help eliminate duplicates. Still remaining -- use canonical URLs for existing subscriptions. I want to do that carefully because I'm still an SQL novice. #BTW, the next things I want to look at for feedBase -- getting a better handle on feeds you've subscribed to that we couldn't read. It's a common feature request. Not exactly sure how I want to do it, but I have ideas. Again, I'm going slow, considering every step carefully, using all the experience we've gotten in 21 years of feed parsing. (Yikes!)#
Another thing. I loved the Wired article about the RSS revival. But I hope it doesn't create the kind of hyper atmosphere we had in the early days. We moved way too fast and created chaos. I want to move slowly and do tech that can last for another 20 years, and hopefully without all the fits and starts and major steps backwards. #
New davefeedread package. Back when I was doing all my server work in Frontier, I had a verb that, given a URL, would parse a feed and return a struct. It did all the work behind a simple interface. When it came time to fix encoding in feedBase, I decided to invest in having it be that simple in Node. I have the new package running inside feedBase now. It seems to work. You're welcome to use it in your projects as well, of course. #
"There's more to life than a little money," Marge said, as if talking to Republicans. "Don't you know that?"#
If anyone from Facebook is listening, now would be a great time to do something kind and generous for the open web. Like for example supporting a subset of Markdown in posts and comments. Most important to support linking. Would do a lot to breathe new life into the web.#
Next items on my feedBase todo. Handle feeds that have special characters, such as feeds in German or Chinese, and merge duplicates, two or more URLs that point to the same feed. I'm going very slow with the changes. This is a long-term project, it's important to get it right. Very different from the first time around with RSS apps when everything was a rush. Also want to take a look at display on phones and tablets. If you have any problems you'd like to report, here's a fresh thread to post in.#
8-minute podcast. I often watch Ari Melber's show on MSNBC, but when they turn to Facebook I get ready to be angry because they screw the story up so badly. Zuck pitched Medvedev and China. He says he wants regulation but has lobbyists that worked against it. Great, you embarrass him, but the net-effect is they continue to destroy the open web. Why don't we work on something that really matters that isn't about Russia or doesn't feel like Watergate. Tech makes messes when companies get dominant. Finding things Facebook is screwing up is easy. But Melber and the rest of the folks in the mainstream press have not found it. Same with Google. #Brain trust query: I'm trying to use the iconv package in Node to translate from ISO-8859-1 to UTF-8. I have a very small test program that fails. Hopefully someone more experienced in Node and encodings can help find the problem. Thanks in advance. 💥#
From the frog-in-boiling-water department. How long before Russian oligarchs murder a journalist in the US? A member of Congress? A US attorney? How will that chill discourse in politics? And assuming their puppet stays in office, because what Congress of either party is going to impeach a president if they might get killed for their vote, how long before social networks are as regulated as those in China? This is likely our future. Not sure we can do anything at this time to prevent it. #However, we're not out of options. We could mass-migrate to low population Republican-dominated states and change the composition of the Senate and the Electoral College.#