Getting Data From the Web (URLConnection via http)

Romeo Ordos · by **Romeo Ordos** » Wed Aug 12, 2009 5:47 pm

And how can I retrieve file size on the server before downloading it?

nachiket · by **nachiket** » Wed Sep 09, 2009 7:07 am

Thank You Plusminus.... really helpful tutorial.

porzino · by **porzino** » Thu Oct 08, 2009 3:24 pm

Hi,

I have try this solution and I have a problem: I only get the first 8K of the web (The size is 300K)

Any ideas?

stevebull · by **stevebull** » Wed Jan 06, 2010 5:15 am

Happy New Year 2010.

Getting Data From the Web (URLConnection via http) was a great tutorial...I got this working quickly and even produces interesting results with my REST script---check it out for yourself. REST is fast for me when I roll my own.

Syntax: [ Download ] [ Hide ]

Using java Syntax Highlighting

 
/* Define the URL we want to load data from. */
 
               URL myURL = new URL(
 
                         // "http://www.anddev.org/images/tut/basic/getdatafromtheweb/loadme.txt"
 
                           // "http://iconic.4feets.com/update/"
 
                           "http://hollywood-usa.mobi/rest/site/index.php?siteID=15"
 
                           );
 
Parsed in 0.031 seconds,  using GeSHi 1.0.8.4

I look forward to reading more from plusminus. Thank you.

Cheers, Steve

sbarnesi · by **sbarnesi** » Fri Jan 29, 2010 10:48 pm

blackpearl wrote:
cs_cckai wrote:Hi,

Actually, I have try your code, but it throw to exception with unknow error, I found that when it run
InputStream is = ucon.getInputStream();
then throw exception.

Can anyone tell me why?

Did you Specify Permission in AndroidManifest file? In the new version, permission needs to be granted.
Please add the following line in AndroidManifest.xml
<uses-permission android:name="android.permission.INTERNET" />

http://code.google.com/android/devel/security.html

You solved my problem! Thanks!

ianwong · by **ianwong** » Tue Feb 23, 2010 11:03 am

Hi, I have the below code using this tutorial code which works fine
in file getting. However, the progress bar does not update (actually
even Toast seems not working) during the file getting period.

As seen, I have log code and log does show the whole progress,
only the progress bar doesn't. The progress bar works fine in general
web surfing.

Can anyone point me which part I'd done wrongly? Thank You!

Syntax: [ Download ] [ Hide ]

Using java Syntax Highlighting

import java.io.FileOutputStream;
 
import java.io.InputStream;
 
import java.net.HttpURLConnection;
 
import java.net.URL;
 
import android.app.Activity;
 
import android.os.Bundle;
 
import android.util.Log;
 
import android.webkit.WebChromeClient;
 
import android.webkit.WebView;
 
import android.webkit.WebViewClient;
 
import android.widget.ProgressBar;
 
import android.widget.Toast;
 
public class Downfile extends Activity {
 
    private static final String fext = ".bin";
 
    private WebView mWebView;
 
    private ProgressBar mProgress;
 
    //private Handler mHandler = new Handler();
 
    @Override
 
    public void onCreate(Bundle icicle){
 
        super.onCreate(icicle);
 
        setContentView(R.layout.downfile);
 
        mWebView = (WebView) findViewById(R.id.webview);
 
        mWebView.getSettings().setJavaScriptEnabled(true);
 
        mWebView.getSettings().setAllowFileAccess(true);
 
        mProgress = (ProgressBar) findViewById(R.id.pb);
 
        mWebView.setWebChromeClient(new WebChromeClient(){
 
                @Override
 
                public void onProgressChanged(WebView v, int progress){
 
                        mProgress.setProgress(progress);
 
                }
 
        });
 
//        mWebView.loadUrl("http://www.google.com");
 
        mWebView.loadUrl(getString(R.string.dl_page));
 
        mWebView.setWebViewClient(new MyWebViewClient());
 
    }
 
    private class MyWebViewClient extends WebViewClient{
 
        @Override
 
        public boolean shouldOverrideUrlLoading(WebView view, String url){
 
                mProgress.setProgress(0);
 
                if (TestNeedFile(url)) testGet(url);
 
                else view.loadUrl(url);
 
                return true;
 
        }
 
        @Override
 
        public void onLoadResource(WebView view, String url){
 
                view.setSaveEnabled(true);
 
                //Toast.makeText(Downfile.this, url, Toast.LENGTH_SHORT).show();
 
        }
 
    }
 
    private boolean TestNeedFile (String link){
 
        String s = link.substring(link.length()-4, link.length());
 
        //Toast.makeText(Downfile.this, s, Toast.LENGTH_SHORT).show();          
 
                if (s.equalsIgnoreCase(fext)) return true;
 
                else return false;
 
    }
 
    private String Getfname (String link){
 
        return link.substring(link.lastIndexOf("/")+1);
 
    }
 
    private void testGet (String link){
 
        try{
 
                String outfile = "/sdcard/mydir/" + Getfname(link);
 
                Toast.makeText(Downfile.this, link, Toast.LENGTH_SHORT).show();         
 
                URL url = new URL(link);
 
                HttpURLConnection uCon = (HttpURLConnection) url.openConnection();
 
                uCon.setRequestMethod("GET");
 
                uCon.setDoOutput(true);
 
                uCon.connect();
 
                FileOutputStream fout = new FileOutputStream(outfile);
 
                InputStream in = uCon.getInputStream();
 
                int lengtotal = uCon.getContentLength();
 
                byte[] Buffer = new byte[1024];
 
                int leng = 0, lengnow=0;
 
                double prog =0;
 
                while ( (leng = in.read(Buffer)) != -1 ){
 
                        fout.write(Buffer,0,leng);
 
                        lengnow+=leng;
 
                        prog = 100 * ((double) lengnow / (double) lengtotal );
 
                        Log.d("File", "Progress = " + Integer.toString((int)prog));
 
                        mProgress.setProgress((int)prog);
 
                }
 
                fout.close();
 
                in.close();
 
        } catch (Exception e){
 
                Log.e("test", "This read is wrong");
 
        }
 
    }
 
}
 
Parsed in 0.045 seconds,  using GeSHi 1.0.8.4

yuriman · by **yuriman** » Wed Mar 17, 2010 3:48 pm

-snip-
forgot to add internet access in manifest

ianwong · by **ianwong** » Thu Mar 18, 2010 3:53 pm

yuriman wrote:-snip-
forgot to add internet access in manifest

nope and I believe if internet access permission does not enabled in manifest, it should result error loading page instead of holding everything during download.

anyway, problem solved by creating new thread for each file it downloads.

yuriman · by **yuriman** » Thu Mar 18, 2010 4:14 pm

ianwong wrote:
yuriman wrote:-snip-
forgot to add internet access in manifest

nope and I believe if internet access permission does not enabled in manifest, it should result error loading page instead of holding everything during download.

anyway, problem solved by creating new thread for each file it downloads.

what?
I solved the problem!
I created a thread couse i want to get mysql stuff to the phone in a table...
And it was the manifest, couse it worked after i added internet access!

DRJTower · by **DRJTower** » Thu Mar 18, 2010 7:06 pm

Is there a tutorial anywhere for doing this in a background service?

I want my activities to be able to communicate with a background service that handle the network (I am using sockets not URLs).

I have already tried putting all my network code in a thread created by the activity (the way I always coded on PC)
but then found out that when an activity is stopped so is any created threads, so I need to do this in a service.

strog · by **strog** » Sat Mar 27, 2010 8:50 am

Hi peeps,
The code is really good.
But what if i would like to write stuff to my server?
I added the following line:
OnputStream os = urlConn.getOutputStream();
and when I have a string (which is in the json format) to be written, nothing actually happens on the other side...
I have the port on 810 set for technical reasons of my project that i want to implement.
In the attachment you will find a .Net Programm that is the server side where I have json objects stored.
Any ideas about why nothing gets to the server?

los93sol · by **los93sol** » Sat Mar 27, 2010 7:50 pm

Hello, firstly, this is a great tutorial, helped me out a lot with the first site I tried to run it against.

I'm having a problem now because the site I'm trying to get the source from requires cookies, and I've been all over trying to get my head around what method I should be using to retrieve the html source from a site that requires cookies be enabled.

Thanks in advance.

imac · by **imac** » Mon Mar 29, 2010 4:09 pm

Thank you for your tutorial. Is helping me a lot.
Tel me please, what if I need to provide an email address and password to get data ?
I am trying to login to Google Adsense and retrieve last data.
Thank you again for your help.

nioupy · by **nioupy** » Sun May 23, 2010 4:05 pm

Berzerk wrote:If anyone is still having problems with character encoding, character set, unicode, etc...

Here is what worked for me:

Syntax: [ Download ] [ Hide ]
Syntax: [ Download ] [ Show ]
Using java Syntax Highlighting
myString = EncodingUtils.getString(baf.toByteArray(), "ISO-8859-1");

Parsed in 0.033 seconds, using GeSHi 1.0.8.4

That line replaced the original line in the GetDataFromWeb tutorial.

The characterset "ISO-8859-1" works perfect for Portuguese and similar languages. For other languages, look up the appropriate characterset and replace it.

Don't forget to include:

Syntax: [ Download ] [ Hide ]
Syntax: [ Download ] [ Show ]
Using java Syntax Highlighting
import org.apache.http.util.EncodingUtils;

Parsed in 0.034 seconds, using GeSHi 1.0.8.4

-Berzerk

Thanks Berzek, you helped me a lot with this line of code !!!
It took me a few tries though to understand that my source was encoded weirdly (windows-1251??) but this plus plusminus code works like a charm !

Thanks both of you

aashishkotagiri · by **aashishkotagiri** » Sat Jul 03, 2010 11:17 am

im getting this "permission denied" error every time i try to acess any url

Syntax: [ Download ] [ Hide ]

Using java Syntax Highlighting

 
 
public class screen2 extends Activity implements OnClickListener {
        
        @Override
        public void onCreate(Bundle instance)
        {
                super.onCreate(instance);
                setContentView(R.layout.view2);
                
                Button bu = (Button)findViewById(R.id.exit);
                bu.setOnClickListener(this);
                
                Button bu1 = (Button)findViewById(R.id.but);
                
                bu1.setOnClickListener(this);
        }
        
        public void onClick(View v)
        {
                TextView tv = (TextView) findViewById(R.id.txt);
                
         if(v.getId()==R.id.exit)
         {
                tv.setText("you clicked exit");
                
                Intent i = new Intent(this,tablelayout.class);
                startActivity(i);
                
         }
         
         if(v.getId()==R.id.but)
         { 
                 String myString = null;
         try {
                
                 URL myURL = new URL("http://www.ssis.co.in/ongc");
                
                 URLConnection ucon = myURL.openConnection();
 
                
                 InputStream is = ucon.getInputStream();
                 BufferedInputStream bis = new BufferedInputStream(is);
                 
                 
                 ByteArrayBuffer baf = new ByteArrayBuffer(50);
                 int current = 0;
                 while((current = bis.read()) != -1){
                         baf.append((byte)current);
                 }
 
                
                 myString = new String(baf.toByteArray());
         } catch (Exception e) {
               
                 myString = e.getMessage();
         }
        
         tv.setText(myString);    
                    
                 //tv.setText("you clicked but"); 
         }
         
         
        }
 
}
 
 
Parsed in 0.041 seconds,  using GeSHi 1.0.8.4

PlusMinus Please help me

Getting Data From the Web (URLConnection via http)

works with my REST

ProgressBar not working during file getting

sending data to the server

Cookie Problems

Re:

Re: Getting Data From the Web (URLConnection via http)

Who is online