Literotica public/private API

[Bramblethorn]

So first of all it's not because a website says something that it has legal value. Youtube says NO to scraping, but legally users can still download these videos, because Google doesn't make the laws.

Second, can you show me where Literotica explicitly indicates scraping is forbidden? Especially for single-person, private use. I read their terms and conditions and it does not seem to be there.

One of the regular posters in the Author's Hangout forums periodically posts site statistics that are compiled by scraping. I've never seen the site owners raise any objection to that.

A while back somebody posted a dump of all the stories from Literotica on Reddit. Copyright violation aside, if they got that via scraping it would've required around a million page loads to compile, which I'd consider "abusive".

OTOH, if they didn't care too much about getting the most up-to-date content, it might've been possible to get that content from Common Crawl without ever touching Literotica's bandwidth. IIRC Lit changed their robots.txt recently and I'm not sure what's getting collected now, but as of a couple of years ago I think they were open to CC and might still be archived there.

 

[OverconfidentSarcasm]

Okay, let's make this simple. Regardless of how people here feel about it:

• I can't find a law that prohibits me from creating archival copies (backups) of publicly and freely accessible content
• I can't find a rule on Literotica prohibiting the download of stories
• I can't even find a rule on Literotica prohibiting the use of adblockers (which would be the most damage a tool like OP is proposing could do to the site: circumventing the ads)

So, since, in general, something is allowed until you find a law that says it's not, I'm gonna claim that OP is good to go regardless of how the authors feel about it.

 

[Bramblethorn]

It's a matter of degrees, though, not kind. Seriously. Right-click and view source, and the whole damned text of the story is in there. All we're doing is splitting hairs; the only significant difference is that a person is/is not looking at the UX fluff and ads. Now, if they turned around and republished them somewhere else, that'd be a different matter, but pretending "text on site" is qualitatively different from "text offline for personal use?" Nah. No more than setting up a DVR to record a show.

Sometimes I'll pull up a story, open the pages in new tabs, and come back to those tabs days or weeks later. I might not even be online at the time, but the content is sitting there until I have time to read it. Hard to see much ethical difference between that and what OP's talking about. And browser features like focus mode will already strip out a lot of the non-story content.

Okay, let's make this simple. Regardless of how people here feel about it:

• I can't find a law that prohibits me from creating archival copies (backups) or publicly and freely accessible content

Unlikely such a law exists, given that things like Wayback and Common Crawl exist very publicly.

On the whole, that's a good thing. Archival sites are valuable because the loss of information is a much bigger problem than the possibility that somebody might have an extra copy of it somewhere. I had a friend who had a writing website where she posted some great stuff; she died suddenly, and in the aftermath renewals didn't go through and the sites were lost. It was a huge relief to us to find that almost all of her work had been saved on Wayback Machine, even if we had to retrieve it one page at a time. I gather that kind of thing happens a lot.

 

[Writer61]

Permission or not, I have the right, both morally and legally.

You really don't.

 

[Writer61]

Clearly, some of you think that downloading a story from this site is not a breach of copyright.

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

 

[ElectricBlue]

Clearly, some of you think that downloading a story from this site is not a breach of copyright.

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

I've got a thousand plus books on my bookshelf, all of which are copyrighted by the author, all of which have been published, and all of which are for my personal use.

How is this scenario any different? Do you want your content read, or not? Gotta dump a copy off Lit 's server onto my device to do that.

If I put new covers on those books and made multiple copies using a scanner, and proceeded to sell them, then I'd be in breach of copyright, and legally and ethically fucked. So would the OP, if he went on and redistributed our stories, but he's not saying he's doing that (he'd be stupid if he did say that).

 

[NoTalentHack]

Clearly, some of you think that downloading a story from this site is not a breach of copyright.

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

As long as they're not publishing it elsewhere, I'm fine with it. I'm not all-in on the Scalzi school of thought that intellectual property is dead, but pretending that being able to right click and make a copy of just about anything you'll ever want to read hasn't changed the landscape is just putting your head in the sand. And as to the moral issues? There isn't one, in my opinion. I'm putting this stuff out here for free; as long as someone isn't turning it around and monetizing it, I don't see a moral issue.

I'll go one further: it's an honor, especially if it's a case where someone isn't just scraping everyone's content and shoving it in an archive. Someone thought what I wrote was so good that they went further than favoriting or following; they made a copy so that they can read it again and again, even if the site dies or I pull it for whatever reason. That's a huge compliment.

 

[OverconfidentSarcasm]

Clearly, some of you think that downloading a story from this site is not a breach of copyright.

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

Of course I'm happy. I put stories on the internet so people can read them. If they download them, it means they want to be able to read them even if their internet is down, Lit goes belly-up, or something else happens. I think that's awesome! If I weren't okay with it, I'd have placed them behind a paywall, as I suggested earlier.

Now, if they download one of my books from Bookapy and then start SHARING the epubs, or take my free stories and put them into bad YouTube videos to make money off, that would be another discussion altogether.

And, again, I can't find a law disallowing downloading the stories. In fact, when I looked, I found the opposite. Like, in the EU, users are even explicitly allowed to UPLOAD their private copies of copyrighted material into cloud storage!

 

[intim8]

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

Absolutely. In fact, anything I put up here, please, PLEASE, pass it around to as many people you can, so long as you keep my name on it, and that I can be found here.

 

[Bramblethorn]

Clearly, some of you think that downloading a story from this site is not a breach of copyright.

Correct. When you posted your stories here you gave Literotica permission to publish them, which implies allowing people to read them via the internet, and it's impossible to do that without downloading them. When I read a Literotica story, my computer makes a connection to Literotica, they transmit a copy of that story to my computer, my computer stores a copy locally and displays that copy to me. It can't show me what it doesn't have.

In ordinary web browsing, that downloaded data is stored in the RAM used by your web browser, and discarded when you close the page. I get the impression you're thinking of "download" as specifically "make a long-term copy on the hard drive". But that's not what the word means.

If that's what you mean by it, then no, I'm not particularly bothered by whether somebody who's reading my story is downloading it to their HD or to to RAM.

But, do you think it's acceptable? Are you happy for somebody to keep a copy of your work without your permission?

Sure. People are welcome to save my stories to their hard drive, print out a copy, all that. I posted my stories here because I wanted them to be read. It would only bother me if they were removing my credit, or using my stuff for personal gain without permission - e.g. making copies and selling them, or reuploading them to their own website and using my stories to draw readers for revenue.

 

[MakerMale]

I've been doing HTML scraping for the past few months, but with the new (pretty terrible btw) authors page layout everything's obviously broken, and the new HTML markup makes it a lot harder to extract useful informations now.

For the API I've found some informations about it on external forums, by reading the source code of some third-party apps, etc.

For instance you can get informations about a user using:
https://literotica.com/api/3/users/<ID>
For author informations (which use the same ID but display different infos):
https://literotica.com/api/3/authors/<ID>
For the list of all stories made by one author:
https://literotica.com/api/3/users/<ID>/stories

Note that these API calls are paginated, I've seen some pagination parameters on github but I haven't reverse-engineered it yet.

Anybody know how to access deeper pages of the API past the first? Only the first 50 submissions are shown on the first story list API page.

 

[MellowJoe]

Anybody know how to access deeper pages of the API past the first? Only the first 50 submissions are shown on the first story list API page.

Yes. I wrote a TamperMonkey script to redo the author stories page. Script here:
https://forum.literotica.com/thread...-author-stories-page-solution-inside.1610478/

To get a list of an author's stories, make this call:
https://literotica.com/api/3/users/AuthorName/series_and_works?params={"page": PAGE, "pageSize": PAGESIZE, "type": "story", "listType": "expanded"}

Set PAGE to 1, and PAGESIZE to 500, and you'll get the first 500 stories. Note: a complete series counts as one story. The parts of the series are all subitems on that entry.

Increment PAGE by 1, and you'll get the next 500. Loop until you got them all.

The values for "type" are:
story - stories
poem - poems
illustra - artwork
audio - audio works

To find out how many stories of each type, make this call:
https://literotica.com/api/3/users/AuthorName?params={"withProfile":false}

The counts are in stories_count, poems_count, illustrations_count, and audios_count.